Eval Definition
Is Helpful
Evaluates whether the response is helpful in solving the user problem or answering their question
This evaluation template assesses whether an AI response is genuinely helpful in addressing the user’s query or request. It evaluates the utility, relevance, and effectiveness of the response in solving the user’s problem or answering their question.
Interface Usage
Python SDK Usage
Example Output
Troubleshooting
If you encounter issues with this evaluation:
- Ensure that both the
input
(user query) andoutput
(AI response) parameters are provided - The helpfulness evaluation works best when the context of the request is clear
- If evaluating complex responses, make sure the entire response is included
- Consider combining with other evaluations like
completeness
orfactual-accuracy
for more comprehensive assessment
Related Evaluations
- completeness: Determines if the response addresses all aspects of the query
- task-completion: Checks if a specific requested task was accomplished
- instruction-adherence: Evaluates if the response follows specific instructions
- is-concise: Assesses whether the response avoids unnecessary verbosity