Create Custom Eval Config

Creates a new custom eval config for a project.

POST https://api.futureagi.com/tracer/custom-eval-config/

Authentication

X-Api-Key API Key Required

Your Future AGI API key used to authenticate requests. You can find and manage your API keys in the Dashboard under Settings.

X-Secret-Key Secret Key Required

Your Future AGI secret key, used alongside the API key for request authentication. This is generated when you create an API key in the Dashboard.

Request body

eval_template string Required

UUID of the eval template to base this config on. Must reference a valid template.

name string Required

Name for the config. Max 255 characters; must be unique within the project among non-deleted configs.

project string Required

UUID of the project to associate this config with.

config object Optional

Configuration dictionary to customize template behavior. Normalized against the template’s config schema; unrecognized fields are ignored.

mapping object Optional

Maps trace span fields to the template’s expected inputs (e.g., {"input": "user_query", "output": "assistant_response"}). Keys with null or empty values are stripped.

filters object Optional

Filter criteria restricting which spans this config applies to.

error_localizer boolean Optional

Enables error localization to identify which output segment caused a failure. Defaults to false.

kb_id string Optional

UUID of a knowledge base file to reference during evaluation.

model string Optional

Evaluation model for scoring. Options: turing_large, turing_small, protect, protect_flash, turing_flash. Defaults to the system default when omitted.

Response

200 OK
id string
UUID of the created custom eval config.

Errors

400 Bad Request Optional

Missing required fields or invalid values.

401 Unauthorized Optional

Invalid or missing API credentials.

500 Internal Server Error Optional

Unexpected server error.

GET /
Authentication
REQUEST 
 
 
  
 
 RESPONSE