Skip to main content
POST
/
model-hub
/
create_custom_evals
/
Create custom evaluation template
curl --request POST \
  --url https://api.futureagi.com/model-hub/create_custom_evals/ \
  --header 'Content-Type: application/json' \
  --header 'X-Api-Key: <api-key>' \
  --header 'X-Secret-Key: <api-key>' \
  --data '{
  "template_type": "Futureagi",
  "name": "custom-eval-template",
  "output_type": "Pass/Fail",
  "required_keys": [
    "input",
    "expected_output"
  ]
}'
{
  "id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
  "template_type": "Llm",
  "name": "<string>",
  "description": "<string>",
  "criteria": "<string>",
  "output_type": "Pass/Fail",
  "required_keys": [
    "<string>"
  ],
  "config": {},
  "tags": [
    "<string>"
  ],
  "created_at": "2023-11-07T05:31:56Z"
}

Authorizations

X-Api-Key
string
header
required

API Key for authentication. Click here to access API Key

X-Secret-Key
string
header
required

Secret Key for authentication. Click here to access Secret Key

Body

application/json
template_type
enum<string>
default:Futureagi
required

Type of custom evaluation template

Available options:
Llm,
Futureagi,
Function
name
string
required

Name of the custom template (lowercase, alphanumeric, hyphens, underscores only)

Maximum length: 255
Example:

"custom-eval-template"

output_type
enum<string>
default:Pass/Fail
required

Type of evaluation output

Available options:
Pass/Fail,
score,
choices
required_keys
string[]
required

Required input keys for the evaluation

Example:
["input", "expected_output"]
description
string

Description of the custom template

tags
string[]

Tags for categorization

Example:
["custom", "test"]
criteria
string

Evaluation criteria

Maximum length: 100000
config
object

Template configuration with allowed keys: model, proxy_agi, visible_ui, reverse_output, config

check_internet
boolean
default:false

Whether to check internet connectivity

choices
object

Choices dictionary when output_type is 'choices'

multi_choice
boolean
default:false

Allow multiple choice selection

Response

Custom evaluation template created successfully

id
string<uuid>
template_type
enum<string>
Available options:
Llm,
Futureagi,
Function
name
string
description
string
criteria
string
output_type
enum<string>
Available options:
Pass/Fail,
score,
choices
required_keys
string[]
config
object
tags
string[]
created_at
string<date-time>