Skip to main content
result = evaluator.evaluate(
    eval_templates="context_relevance",
    inputs={
        "context": "Honey never spoils because it has low moisture content and high acidity, creating an environment that resists bacteria and microorganisms. Archaeologists have even found pots of honey in ancient Egyptian tombs that are still perfectly edible.",
        "input": "Why doesn't honey go bad?"
    },
    model_name="turing_flash"
)

print(result.eval_results[0].output)
print(result.eval_results[0].reason)
Input
Required InputTypeDescription
contextstringThe context provided to the model
inputstringThe input provided to the model
Output
FieldDescription
ResultReturns a score, where higher scores indicate more relevant context
ReasonProvides a detailed explanation of the context relevance assessment
What to do when Context Relevance is Low When context relevance is low, the first step is to identify which parts of the context are either irrelevant or insufficient to address the query effectively. If critical information is missing, additional details should be incorporated to ensure completeness. At the same time, any irrelevant content should be removed or refined to improve focus and alignment with the query. Implementing mechanisms to enhance context-query alignment can further strengthen relevance, ensuring that only pertinent information is considered. Additionally, optimising context retrieval processes can help prioritise relevant details, improving overall response accuracy and coherence.
Differentiating Context Relevance with Similar Evals
  1. Context Adherence: It measures how well responses stay within the provided context while Context Relevance evaluates the sufficiency and appropriateness of the context.
  2. Completeness: Completeness evaluates if the response completely answers the query, while Context Relevance focuses on the context’s ability to support a complete response.
  3. Context Similarity: It compares similarity between provided and expected context, that is, it measures how closely the context matches expected information, while Context Relevance assesses if the context is sufficient and appropriate for the query.
⌘I