Run New Evals on Executions

Your Future AGI API key used to authenticate requests. You can find and manage your API keys in the Dashboard under Settings.

Your Future AGI secret key, used alongside the API key for request authentication. This is generated when you create an API key in the Dashboard.

UUID of the test run containing the executions to evaluate.

Array of test execution UUIDs to evaluate. Required when select_all is false. Only COMPLETED executions are eligible.

When true, evaluates all completed executions, ignoring test_execution_ids. Defaults to false.

Array of evaluation configuration UUIDs to run on the selected executions.

When true, also evaluates tool usage by the agent. Defaults to false.

Validation error. Common causes: missing eval_config_ids, neither select_all nor test_execution_ids provided, no completed executions found.

{"error": "Either 'select_all' must be True or 'test_execution_ids' must be provided"}

Or when no completed executions exist:

{"error": "No test executions found to run evaluations on."}

Or when executions are not completed:

{"error": "Only test executions with COMPLETED status can have new evaluations run on them."}

Missing or invalid X-Api-Key or X-Secret-Key headers.

No test run found with the specified run_test_id.

{"detail": "No RunTest matches the given query."}

Unexpected server error.

{"error": "Failed to run evaluations: <message>"}