Add evaluation configurations

Adds evaluation configurations to a test run.

POST https://api.futureagi.com/simulate/run-tests/{run_test_id}/eval-configs/

Authentication

X-Api-Key API Key Required

Your Future AGI API key used to authenticate requests. You can find and manage your API keys in the Dashboard under Settings.

X-Secret-Key Secret Key Required

Your Future AGI secret key, used alongside the API key for request authentication. This is generated when you create an API key in the Dashboard.

Path parameters

run_test_id UUID Required

UUID of the test run to add evaluation configurations to.

Request body

evaluations_config array of objects Required

Array of evaluation configuration objects.

template_id string Required

UUID of the evaluation template to use.

name string Required

Name for this evaluation configuration. Must be unique within the test run.

config object

Template-specific configuration parameters.

mapping object

Maps test execution data fields to the evaluation template’s expected inputs.

filters object

Filter criteria to restrict which test results are evaluated.

error_localizer boolean

Enables granular error localization on evaluation failures. Defaults to false.

model string

Model to use for running this evaluation.

Response

201 Created
message string
Confirmation of successful addition.
created_eval_configs array of objects

Array of the newly created evaluation config objects. Each object contains: id, name, config, mapping, filters, error_localizer, model, status, template_id.

run_test_id string
UUID of the test run the configs were added to.
warnings array of strings Optional

Present only when some configs in the request failed to create while others succeeded. Lists per-item error messages.

Errors

400 Bad Request Optional

Invalid or missing fields such as a non-existent template_id, duplicate name, or malformed config/mapping.

401 Unauthorized Optional

Missing or invalid X-Api-Key or X-Secret-Key headers.

404 Not Found Optional

No test run found with the specified run_test_id.

500 Internal Server Error Optional

Unexpected server error.

GET /
Authentication
REQUEST 
 
 
  
 
 RESPONSE