Context Sufficiency

Definition

Determines whether provided context contains adequate information to answer a specific query. This evaluation uses language models to assess the completeness and relevance of context for query resolution.

A Passed evaluation indicates the context contains sufficient information to answer the query, while a Failed evaluation suggests inadequate or missing information.

Calculation

System analyses the query, examines the context, and selects the appropriate model from the available options. Next, in the sufficiency assessment phase, the system evaluates the relevance of the context, checks for completeness of information, and verifies alignment between the query and the provided context.

Eval returns a binary Pass/Fail output based on the analysis of context sufficiency, determining whether the available information is adequate to address the query.

What to do when Context Sufficiency Fails

When context sufficiency fails, a thorough context review is necessary to ensure completeness, relevance, and overall quality. This involves assessing whether the provided context contains all necessary information and is appropriately aligned with the query.

Additionally, a query analysis should be conducted to evaluate clarity, scope, and alignment with the available context. If discrepancies exist, refining the context or adjusting the query can help improve sufficiency and ensure a more accurate evaluation.

Differentiating Context Sufficiency with Context Relevance

Context Sufficiency focuses on ensuring that the provided context contains all necessary information to address the query, while Context Relevance assesses how pertinent the context is to the input.

In terms of required inputs, Context Sufficiency relies on both the query and context, whereas Context Relevance evaluates the relationship between the context and input.

The output formats also differ, Context Sufficiency produces a Pass/Fail result based on completeness, whereas Context Relevance generates a score-based output reflecting the degree of relevance.