Context Adherence
Measures how well a response stays within the provided context, flagging statements not grounded in it to catch hallucination.
Context Adherence checks whether a response sticks to the information given in its context, or introduces claims the context doesn’t support. Run it to catch hallucination in RAG and grounded-generation pipelines.
What it does
Context Adherence is an LLM-as-Judge eval. It reads the context and the generated output, then scores how much of the output is actually grounded in that context.
Input
| Required Input | Type | Description |
|---|---|---|
context | string | The context provided to the model |
output | string | The output generated by the model |
Output
| Field | Type | Description |
|---|---|---|
result.score | float (0–1) | Higher scores (closer to 1) indicate stronger adherence to the context |
result.reason | string | A plain-language explanation of the score |
Run it from code
Call evaluate() with the template name and the eval’s required inputs. It returns the score and the reason.
Note
Before running: install the SDK and set FI_API_KEY / FI_SECRET_KEY. The model argument in the snippets is the evaluator model Future AGI uses to run the eval; turing_flash is a fast default.
from fi.evals import evaluate
result = evaluate(
"context_adherence",
context="Honey never spoils because it has low moisture content and high acidity, creating an environment that resists bacteria and microorganisms. Archaeologists have even found pots of honey in ancient Egyptian tombs that are still perfectly edible.",
output="Honey doesn't spoil because its low moisture and high acidity prevent the growth of bacteria and other microbes.",
model="turing_flash",
)
print(result.score)
print(result.reason)import { evaluate } from "@future-agi/ai-evaluation";
const result = await evaluate(
"context_adherence",
{
context: "Honey never spoils because it has low moisture content and high acidity, creating an environment that resists bacteria and microorganisms. Archaeologists have even found pots of honey in ancient Egyptian tombs that are still perfectly edible.",
output: "Honey doesn't spoil because its low moisture and high acidity prevent the growth of bacteria and other microbes."
},
{ modelName: "turing_flash" }
);
console.log(result);When to use
Run Context Adherence wherever a response is supposed to be grounded in a specific context and you need to catch claims that aren’t.
- Text, audio, image, and chat outputs generated from a fixed context
- RAG and retrieval pipelines, to confirm the answer doesn’t go beyond the retrieved chunks
- Hallucination checks, where you need to flag statements the context doesn’t support
What to do when Context Adherence fails
When context adherence is low, start by identifying statements that are not supported by the provided context and checking for implicit versus explicit information to assess potential misinterpretations.
Reviewing how the context is processed can help pinpoint inconsistencies. If necessary, expand context coverage to fill in gaps, clarify ambiguous details, and add missing relevant information.
To improve adherence, implement stricter context binding, integrate fact-checking mechanisms, and enhance overall context processing.
Questions & Discussion