Eval Definition
Is Compliant
Evaluates whether content follows guidelines, standards, and acceptable use policies.
Evaluation Using Interface
Input:
- Required Inputs:
- input: The text content to evaluate for compliance.
Output:
- Result: Returns ‘Passed’ if the content is compliant with guidelines and policies, ‘Failed’ if it’s non-compliant.
- Reason: A detailed explanation of why the content was classified as compliant or non-compliant.
Evaluation Using Python SDK
Click here to learn how to setup evaluation using the Python SDK.
Input:
- Required Inputs:
- input:
string
- The text content to evaluate for compliance.
- input:
Output:
- Result: Returns a list containing ‘Passed’ if the content is compliant with guidelines and policies, or ‘Failed’ if it’s non-compliant.
- Reason: Provides a detailed explanation of the evaluation.
Example Output:
What to do If you get Undesired Results
If the content is evaluated as non-compliant (Failed) and you want to improve it:
- Remove any potentially offensive, harmful, or discriminatory language
- Avoid content that could be interpreted as promoting illegal activities
- Ensure respectful and professional tone throughout
- Remove mentions of restricted topics based on relevant guidelines
- Avoid aggressive, threatening, or harassing language
- Eliminate content that could be interpreted as deceptive or manipulative
- Check for privacy violations or sharing of sensitive information
- Consider cultural sensitivities and avoid stereotyping
Comparing Is Compliant with Similar Evals
- Content Safety Violation: Is Compliant provides a broader assessment of guideline adherence, while Content Safety Violation focuses specifically on detecting harmful, unsafe content.
- Is Harmful Advice: Is Compliant evaluates overall policy adherence, whereas Is Harmful Advice specifically identifies potentially dangerous recommendations.
- Toxicity: Is Compliant evaluates general policy compliance, while Toxicity specifically measures harmful or offensive language.