Answer completeness evaluation assesses whether a model’s response fully and adequately answers the given query. This evaluation is crucial for ensuring that responses address all aspects of the user’s question without leaving important elements unanswered.

​
Configuration

The evaluation requires the following configuration:

ParameterDescription
modelThe model to be used for evaluation
from fi.evals import AnswerCompleteness

answer_eval = AnswerCompleteness(config={"model": "gpt-4o-mini"})

​
Test Case Setup

The evaluation requires both the query and the response to evaluate:

from fi.testcases import LLMTestCase

test_case = LLMTestCase(
    query="What are the three primary colors?",
    response="The three primary colors are red, blue, and yellow."
)

​
Client Setup

Initialize the evaluation client with your API credentials:

from fi.evals import EvalClient

evaluator = EvalClient(
    fi_api_key="your_api_key", 
    fi_secret_key="your_secret_key"
)

​
Complete Example

from fi.evals import AnswerCompleteness, EvalClient
from fi.testcases import LLMTestCase

# Initialize the answer completeness evaluator
answer_eval = AnswerCompleteness(config={"model": "gpt-4o-mini"})

# Create a test case
test_case = LLMTestCase(
    query="What are the three primary colors?",
    response="The three primary colors are red, blue, and yellow."
)

# Run the evaluation
evaluator = EvalClient(fi_api_key="your_api_key", fi_secret_key="your_secret_key")
result = evaluator.evaluate(answer_eval, test_case)
print(result)  # Will return Pass if response completely answers the query
Context Contains Enough InformationGrading Criteria