Definition

Evaluates the quality of the context retrieved for generating a response. This evaluation ensures that the context used is relevant and sufficient to produce an accurate and coherent output.

A high score indicates that the context is well-suited for the task, while a low score suggests inadequacies in the context.

Calculation

The evaluation process begins with configuring the input, output, and context, specifying the evaluation criteria that will guide the assessment. Context quality analysis is then performed to determine the relevance and sufficiency of the context in relation to the input and output, ensuring it supports the generation of a coherent and accurate response. A score is assigned based on this analysis and compared against predefined criteria to assess whether the context meets expected standards.

What to do if Eval Context Retrieval Quality is Low

If the evaluation returns a low score, the criteria should be reviewed to ensure they are well-defined, relevant, and aligned with the evaluation’s objectives. Adjustments may be necessary to enhance clarity and comprehensiveness. The context should also be analysed for relevance and sufficiency, identifying any gaps or inadequacies and refining it as needed to better support the output.

Differentiating Eval Context Retrieval Quality with Context Adherence

Eval Context Retrieval Quality and Context Adherence serve different purposes. Eval Context Retrieval Quality assesses the overall quality and relevance of the retrieved context, ensuring it is sufficient and appropriate for generating a response. In contrast, Context Adherence focuses on whether the response strictly adheres to the provided context, preventing the introduction of external information.