Context evaluation encompasses multiple metrics to assess how effectively context is used in generating responses. These evaluations help validate context retrieval, utilization, and adherence in RAG (Retrieval-Augmented Generation) systems.

​
Available Context Evaluations

Evaluation TypeDescription
ContextRetrievalAssesses quality of retrieved context
ContextAdherenceMeasures how well responses stay within provided context

​
Required Parameters

ParameterDescriptionRequired
inputThe input query or promptYes
outputThe generated responseYes
contextThe context used to generate the outputYes

​
Configuration

The evaluation accepts the following configuration:

ParameterDescriptionRequiredDefault
check_internetWhether to verify facts against internet sourcesNofalse
modelThe LLM model to use for evaluationYes-

​
Usage Examples

​
Context Adherence Example

from fi.evals import ContextAdherence
from fi.testcases import LLMTestCase

# Initialize context adherence evaluator
adherence_eval = ContextAdherence(
    config = {
        "check_internet": False
    }
)

# Create test case
test_case = TestCase(
    input="What is the capital of France?",
    output="Paris is the capital city of France.",
    context="Paris is the capital and largest city of France, situated in the north-central part of the country."
)

result = evaluator.evaluate(adherence_eval, test_case)
print(result)  # Returns a score indicating adherence to context
Content ValidationDeterministic Evaluation