Ground Truth Match

Checks if a model's output matches a provided ground truth answer, judging semantic equivalence rather than exact string matching.

Ground Truth Match checks whether a model’s output matches a known correct answer. Run it when you have a ground truth value and need a pass or fail verdict rather than a similarity score.

What it does

Ground Truth Match is an LLM-as-Judge eval. It reads the generated value and the expected value, then judges whether they are equivalent in meaning, not just in wording.

Input

Required Input	Type	Description
`generated_value`	`string`	The model-generated output to be evaluated
`expected_value`	`string`	The ground-truth reference output

Output

Field	Type	Description
Result	`Pass / Fail`	Pass means the generated output matches or is equivalent to the expected ground truth, Fail means they differ in meaning, correctness, or format
Reason	`string`	A plain-language explanation of the match assessment

Run it from code

Call evaluate() with the template name and the eval’s required inputs. It returns the score and the reason.

Note

Before running: install the SDK and set FI_API_KEY / FI_SECRET_KEY. The model argument in the snippets is the evaluator model Future AGI uses to run the eval; turing_flash is a fast default.

from fi.evals import evaluate

result = evaluate(
    "ground_truth_match",
    generated_value="The capital of France is Paris.",
    expected_value="Paris is the capital of France.",
    model="turing_flash",
)

print(result.score)
print(result.reason)

import { evaluate } from "@future-agi/ai-evaluation";

const result = await evaluate(
  "ground_truth_match",
  {
    generated_value: "The capital of France is Paris.",
    expected_value: "Paris is the capital of France."
  },
  { modelName: "turing_flash" }
);

console.log(result);

When to use

Run Ground Truth Match wherever you have a known-correct answer and need a clear pass or fail call instead of a graded score.

Text and audio outputs with a single correct answer, such as fact lookups or QA pairs
Regression checks where a known ground truth exists and any deviation in meaning should fail
Cases where paraphrasing is acceptable but factual or formatting drift is not

What to do when Ground Truth Match fails

Review the generated output for factual errors or missing information. Check if the format of the generated output matches what was expected.

Ensure the model has access to the correct context to produce the right answer, and consider whether the expected value allows for paraphrasing or requires an exact match.

Was this page helpful?

Questions & Discussion