Create a New Test Run

Creates and configures a new test run, associating it with scenarios, an agent definition, and detailed evaluation configurations.

          Click "Try It" to send a request
        

Authentication

This endpoint requires authentication. Include your API key in the Authorization header:

Authorization: Bearer YOUR_API_KEY

Request Body

• name (required): string A unique name for the test run.
• description: string An optional description for the test run.
• scenarioIds (required): array of string A list of scenario UUIDs to be included in this test run.
• agentDefinitionId (required): string The UUID of the agent definition to be tested.
• agentVersion: string The specific UUID of the agent version to be tested. If not provided, the active version will be used.
• evalConfigIds: array of string A list of existing evaluation configuration UUIDs to associate with this test run.
• evaluationsConfig: array of any A list of new, detailed evaluation configurations to create and associate with this test run.
• datasetRowIds: array of string A list of specific dataset row UUIDs to test against.
• enableToolEvaluation: boolean Flag to enable tool evaluation for this test run.

Example

{
  "name": "your-name",
  "description": "your-description",
  "scenarioIds": [],
  "agentDefinitionId": "your-agentDefinitionId",
  "agentVersion": "your-agentVersion",
  "evalConfigIds": [],
  "evaluationsConfig": [],
  "datasetRowIds": [],
  "enableToolEvaluation": true
}

Responses

201

The test run was created successfully.

400

Bad Request. The request payload is invalid.

401

Unauthorized. Authentication credentials were not provided.

404

Not Found. The user’s organization or other specified resources could not be determined.

500

Internal Server Error. An unexpected error occurred.

Code Examples

cURL

curl -X POST "https://api.futureagi.com/simulate/run-tests/create/" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"name": "your-name", "description": "your-description", "scenarioIds": [], "agentDefinitionId": "your-agentDefinitionId", "agentVersion": "your-agentVersion", "evalConfigIds": [], "evaluationsConfig": [], "datasetRowIds": [], "enableToolEvaluation": true}'

Python

import requests

url = "https://api.futureagi.com/simulate/run-tests/create/"
headers = {
    "Authorization": "Bearer YOUR_API_KEY",
    "Content-Type": "application/json"
}
data = {
    "name": "your-name",
    "description": "your-description",
    "scenarioIds": [],
    "agentDefinitionId": "your-agentDefinitionId",
    "agentVersion": "your-agentVersion",
    "evalConfigIds": [],
    "evaluationsConfig": [],
    "datasetRowIds": [],
    "enableToolEvaluation": true
}

response = requests.post(url, headers=headers, json=data)
print(response.json())

JavaScript

const response = await fetch("https://api.futureagi.com/simulate/run-tests/create/", {
  method: "POST",
  headers: {
    "Authorization": "Bearer YOUR_API_KEY",
    "Content-Type": "application/json"
  },
  body: JSON.stringify({
    "name": "your-name",
    "description": "your-description",
    "scenarioIds": [],
    "agentDefinitionId": "your-agentDefinitionId",
    "agentVersion": "your-agentVersion",
    "evalConfigIds": [],
    "evaluationsConfig": [],
    "datasetRowIds": [],
    "enableToolEvaluation": true
})
});

const data = await response.json();
console.log(data);