- Doesn’t follow the user’s instructions.
- Introduces information that isn’t part of the given context.
- Strays into unrelated topics or makes unsupported claims.
1. Prompt/Instruction Adherence
Measures how closely an output follows given prompt instructions, checking for completion of requested tasks and adherence to specified constraints or formats. Click here to read the eval definition of Prompt/Instruction Adherencea. Using Interface
Inputs Required:- Output: The column that has response generated by the AI system.
b. Using SDK
Export your API key and Secret key into your environment variables.
2. Context Adherence
Evaluates how well responses stay within the provided context by measuring if the output contains any information not present in the given context. Click here to read the eval definition of Context Adherencea. Using Interface
Inputs Required:- Output: The response generated by the AI system.
- Context: The background information or source material the response should be based on.