Completeness
Evaluates whether a response fully addresses every aspect of the input query, flagging gaps in coverage.
Completeness checks whether a response covers everything the query asked for, or leaves parts of it unanswered. Run it wherever a partial answer is as bad as a wrong one.
What it does
Completeness is an LLM-as-Judge eval. It reads the input query and the generated output, then scores how fully the output addresses every aspect of the query.
Input
| Required Input | Type | Description |
|---|---|---|
input | string | User query provided to the model |
output | string | Model generated response |
Output
| Field | Type | Description |
|---|---|---|
| Result | score | Higher scores indicate more complete content relative to the input |
| Reason | string | A plain-language explanation of the completeness assessment |
Run it from code
Call evaluate() with the template name and the eval’s required inputs. It returns the score and the reason.
Note
Before running: install the SDK and set FI_API_KEY / FI_SECRET_KEY. The model argument in the snippets is the evaluator model Future AGI uses to run the eval; turing_flash is a fast default.
from fi.evals import evaluate
result = evaluate(
"completeness",
input="Why doesn't honey go bad?",
output="Honey doesn't spoil because its low moisture and high acidity prevent the growth of bacteria and other microbes.",
model="turing_flash",
)
print(result.score)
print(result.reason)import { evaluate } from "@future-agi/ai-evaluation";
const result = await evaluate(
"completeness",
{
input: "Why doesn't honey go bad?",
output: "Honey doesn't spoil because its low moisture and high acidity prevent the growth of bacteria and other microbes."
},
{ modelName: "turing_flash" }
);
console.log(result);When to use
Run Completeness wherever a response needs to answer the full question, not just part of it.
- Text, audio, and chat outputs that respond to multi-part or multi-step queries
- RAG and retrieval pipelines, to confirm the answer uses all the relevant retrieved material
- Support and Q&A flows, where an unanswered sub-question sends the user back for another round
What to do when Completeness fails
Determine which aspects of the query have not been fully addressed and identify any gaps or incomplete sections that require additional information.
Enhancing the response involves adding missing details to ensure it’s comprehensive and refining the content to cover all aspects of the query.
To improve completeness in the long term, implement mechanisms that align responses more closely with query requirements and enhance the response generation process to prioritize completeness.
Questions & Discussion