Skip to main content
result = evaluator.evaluate(
    eval_templates="is_good_summary", 
    inputs={
        "input": "Honey never spoils because it has low moisture content and high acidity, creating an environment that resists bacteria and microorganisms. Archaeologists have even found pots of honey in ancient Egyptian tombs that are still perfectly edible.",
        "output": "Honey doesn't spoil because its low moisture and high acidity prevent the growth of bacteria and other microbes."
    },
    model_name="turing_flash"
)

print(result.eval_results[0].metrics[0].value)
print(result.eval_results[0].reason)
Input
Required InputTypeDescription
inputstringThe original source content.
outputstringGenerated summary by the model to be evaluated.
|
Output
FieldDescription
ResultReturns Passed if the summary effectively captures the key information, or Failed if it doesn’t.
ReasonProvides a detailed explanation of why the summary was deemed good or poor.

​
What to do If you get Undesired Results

If the summary is evaluated as poor (Failed) but you want to improve it:
  • Ensure all key points from the original source content are included
  • Maintain the core meaning and intent of the original source content
  • Remove unnecessary details but keep essential information
  • Keep the summary concise while preserving important context
  • Avoid adding new information not present in the original source content
  • Use clear language that accurately represents the original content

​
Comparing Is Good Summary with Similar Evals

  • Summary Quality: While Is Good Summary provides a binary assessment (Passed/Failed), Summary Quality might offer more granular ratings of summary effectiveness.
  • Completeness: Is Good Summary focuses on the overall effectiveness of a summary, whereas Completeness specifically measures whether all required information is included.
⌘I