eval_name
: The evaluations to run on the spanstype
: Specifies where to apply the evaluationvalue
: Identifies the kind of span to evaluatemapping
: Contains mapping of the required inputs of the eval Learn more →custom_eval_name
: Custom name to assign the eval tagmodel
: Model name to be assigned especially incase of future-agi evalsmapping
attribute is a crucial component that connects eval requirements with your data. Here’s how it works:
context
and output
keys.
output
key required by the eval will use data from this span attribute llm.output_messages.0.message.content
context
input will use data from this span attribute llm.input_messages.1.message.content
string
- The text to be evaluated for content moderationstring
- The context provided to the AI system.string
- The output generated by the AI system.string
- The context provided to the AI system.string
- The input to the AI system.string
- The input to the AI system.string
- The output generated by the AI system.string
- The input to be evaluated for PIIstring
- the input to be evaluated for toxicitystring
- The input to be evaluated for tonestring
- The input to be evaluated for sexismstring
- The input to the AI systemstring
- The output generated by the AI system.string
- The input to be evaluatedstring
- The text to be evaluatedstring
- The text to be evaluatedstring
- The text to be evaluatedstring
- The text to be evaluatedstring
- The input to the AI systemstring
- The output generated by the AI systemstring
- The context provided to the AI systemstring
- The input provided to the AI systemstring
- The output generated by the AI systemstring
- The context provided to the AI systemstring
- The input to the AI systemstring
- The translated output generated by the AI systemstring
- The input to be evaluated for cultural sensitivitystring
- The input to be evaluated for biasstring
- The input to the AI systemstring
- The output generated by the AI systemstring
- The input to the AI systemstring
- The output generated by the AI systemstring
- The URL of the audio to be evaluatedstring
- The output generated by the AI systemstring
- The URL of the audio to be evaluatedstring
- The input to the AI systemstring
- The output generated by the AI systemstring
- The context provided to the AI systemstring
- The input to the AI systemstring
- The output generated by the AI systemstring
- The context provided to the AI systemstring
- The input to the AI systemstring
- The output generated by the AI systemstring
- The input providedstring
- The input providedstring
- The input providedstring
- The input providedstring
- The input providedstring
- The input providedstring
- The input to be evaluated for concisenessstring
- The user’s questionstring
- The response to be evaluatedstring
- The input to be evaluated for code validitystring
- The output to be evaluatedstring
- The expected answer to compare againststring
- The harmful/sensitive querystring
- The model’s responsestring
- The original input/referencestring
- The response to be evaluated for hallucinationsstring
- The input to be evaluated for harmful therapeutic guidancestring
- The input to be evaluated for clinical tone appropriatenessstring
- The input to be evaluated for harmful advicestring
- The input to be evaluated for content moderationstring
- The source materialstring
- The summary to be evaluatedstring
- The source/context materialstring
- The output to be evaluated for factual consistencystring
- The input to be evaluated for compliancestring
- The input to be evaluated for tone formalitystring
- The user’s requeststring
- The function call to be evaluatedstring
- The user’s requeststring
- The model’s response to be evaluatedstring
- The user’s requeststring
- The model’s response to be evaluatedstring
- The reference answerstring
- The model outputstring
- The reference answerstring
- The model outputstring
- The input text to be evaluatedstring
- The output to be evaluatedstring
- The reference setstring
- The retrieved setstring
- Model-generated output to be evaluatedstring
- Reference string against which the output is comparedstring
- Model-generated output to be evaluatedstring
- Reference string against which the output is comparedstring
- Model-generated output to be evaluatedstring
- Reference string against which the output is comparedstring
- Model-generated output to be evaluatedstring
or List[string]
- Reference phrases or keywordsstring
- The input image to be evaluated