Ensuring that a query is supported by sufficient context is critical for accurate and reliable responses. Inadequate context can lead to incomplete, ambiguous, or incorrect answers, impacting decision-making and information retrieval.

This eval determines whether the provided content contains enough information to accurately address a given query.

Click here to read the eval definition of Context Sufficiency

​
a. Using Interface

Inputs Required:

  • Query: The question or request requiring an answer.
  • Context: The background information used to generate the response.

Configuration Parameters:

  • Model: LLM model to be used here to analyze context sufficiency.

Output

  • 1: The context contains sufficient information to fully answer the query.
  • 0: The context lacks key details, requiring additional information or refinement.

​
b. Using SDK

from fi.evals import EvalClient
from fi.evals.templates import ContextSufficiency
from fi.testcases import TestCase

evaluator = EvalClient(
    fi_api_key="your_api_key",
    fi_secret_key="your_secret_key",
    fi_base_url="https://api.futureagi.com"
)

test_case = TestCase(
    query="What is the capital of France?",
    context="Paris has been France's capital since 987 CE."
)

template = ContextSufficiency(config={
    "model": "gpt-4o-mini"
})

response = evaluator.evaluate(eval_templates=[template], inputs=[test_case])

print(f"Score: {response.eval_results[0].metrics[0].value}")
print(f"Reason: {response.eval_results[0].reason}")
Evaluate SummaryOverview