Create Run Test

Create a new test run with scenarios, agent definition, eval configs, and optional tool evaluation. Returns the run test UUID, name, and associated scenario list.

POST https://api.futureagi.com/simulate/run-tests/create/

Authentication

X-Api-Key API Key Required

Your Future AGI API key used to authenticate requests. You can find and manage your API keys in the Dashboard under Settings.

X-Secret-Key Secret Key Required

Your Future AGI secret key, used alongside the API key for request authentication. This is generated when you create an API key in the Dashboard.

Request body

name string Required

Name for the test run. Must be unique within your organization and cannot exceed 255 characters.

description string Optional

Optional free-text description of the test run.

scenario_ids array of string Required

Array of scenario UUIDs to execute against. Must contain at least one valid scenario ID.

agent_definition_id string Required

UUID of the agent definition to evaluate.

eval_config_ids array of string Optional

Array of existing evaluation configuration UUIDs to associate with this test run.

evaluations_config array of objects Optional

Array of inline evaluation configuration objects to create and associate. Each object must include template_id, name, config, and mapping.

dataset_row_ids array of string Optional

Array of dataset row UUIDs to restrict execution to specific data entries. If omitted, all rows are included.

enable_tool_evaluation boolean Optional

When true, evaluates correctness of tool calls made by the agent. Defaults to false.

replay_session_id string Optional

Optional UUID of a session to replay. When provided, execution replays the specified session.

Response

201 Created
id string

UUID of the newly created test run.

name string

Name of the test run.

description string

Description of the test run, or empty string if none provided.

agent_definition string

UUID of the associated agent definition.

agent_version string

UUID of the specific agent version, or null if using the active version.

agent_definition_detail object

Detailed agent definition object, or null.

source_type string

Source type identifier (e.g. "agent_definition").

source_type_display string

Human-readable source type label (e.g. "Agent Definition").

scenarios array of string

Array of linked scenario UUIDs.

scenarios_detail array of objects

Array of detailed scenario objects.

dataset_row_ids array of string

Array of dataset row UUIDs associated with this test run.

simulator_agent string

UUID of the simulator agent, or null.

simulator_agent_detail object

Detailed simulator agent object, or null.

simulate_eval_configs array of string

Array of evaluation configuration UUIDs.

simulate_eval_configs_detail array of objects

Array of detailed evaluation configuration objects.

evals_detail array of objects

Array of detailed evaluation result objects.

organization string

UUID of the owning organization.

enable_tool_evaluation boolean

Whether tool evaluation is enabled.

created_at string

ISO 8601 creation timestamp.

updated_at string

ISO 8601 last-modified timestamp.

last_run_at string

ISO 8601 timestamp of the most recent execution, or null.

deleted boolean

Whether the test run has been soft-deleted.

deleted_at string

ISO 8601 timestamp of soft-deletion, or null.

Errors

400 Bad Request Optional

Invalid or missing required fields, such as empty scenarioIds, invalid UUIDs, or malformed evaluationsConfig.

401 Unauthorized Optional

Missing or invalid X-Api-Key or X-Secret-Key headers.

404 Not Found Optional

Organization not found, or one or more referenced resources (agent definition, scenarios, eval configs) do not exist.

500 Internal Server Error Optional

Unexpected server error. Contact support if it persists.

GET /
Authentication
REQUEST 
 
 
  
 
 RESPONSE