Completeness

Evaluates whether the response fully addresses the input query. This evaluation is crucial for ensuring that the generated response is comprehensive and leaves no aspect of the query unanswered. 

result = evaluator.evaluate(
    eval_templates="completeness",
    inputs={
        "input": "Why doesn't honey go bad?",
        "output": "Honey doesn't spoil because its low moisture and high acidity prevent the growth of bacteria and other microbes."
    },
    model_name="turing_flash"
)

print(result.eval_results[0].output)
print(result.eval_results[0].reason)
import { Evaluator, Templates } from "@future-agi/ai-evaluation";

const evaluator = new Evaluator();

const result = await evaluator.evaluate(
  "completeness",
  {
    input: "Why doesn't honey go bad?",
    output: "Honey doesn't spoil because its low moisture and high acidity prevent the growth of bacteria and other microbes."
  },
  {
    modelName: "turing_flash",
  }
);

console.log(result);
Input
Required InputTypeDescription
inputstringUser query provided to the model
outputstringmodel generated response
Output
FieldDescription
ResultReturns a numeric score, where higher scores indicate more complete content relative to the input
ReasonProvides a detailed explanation of the completeness assessment

What to do when Completeness is Low

Determine which aspects of the query have not been fully addressed and identify any gaps or incomplete sections that require additional information.

Enhancing the response involves adding missing details to ensure it is comprehensive and refining the content to cover all aspects of the query.

To improve completeness in the long term, implementing mechanisms that align responses more closely with query requirements and enhancing the response generation process to prioritise completeness can help ensure more thorough and accurate outputs.

Was this page helpful?

Questions & Discussion