Input | |||
---|---|---|---|
Required Input | Type | Description | |
input | string | The original source content. | |
output | string | Generated summary by the model to be evaluated. |
Output | ||
---|---|---|
Field | Description | |
Result | Returns Passed if the summary effectively captures the key information, or Failed if it doesn’t. | |
Reason | Provides a detailed explanation of why the summary was deemed good or poor. |
What to do If you get Undesired Results
If the summary is evaluated as poor (Failed) but you want to improve it:- Ensure all key points from the original source content are included
- Maintain the core meaning and intent of the original source content
- Remove unnecessary details but keep essential information
- Keep the summary concise while preserving important context
- Avoid adding new information not present in the original source content
- Use clear language that accurately represents the original content
Comparing Is Good Summary with Similar Evals
- Summary Quality: While Is Good Summary provides a binary assessment (Passed/Failed), Summary Quality might offer more granular ratings of summary effectiveness.
- Completeness: Is Good Summary focuses on the overall effectiveness of a summary, whereas Completeness specifically measures whether all required information is included.