​
Evaluation Using Interface

Input:

  • Required Inputs:
    • input: The text content to evaluate for compliance.

Output:

  • Result: Returns ‘Passed’ if the content is compliant with guidelines and policies, ‘Failed’ if it’s non-compliant.
  • Reason: A detailed explanation of why the content was classified as compliant or non-compliant.

​
Evaluation Using Python SDK

Click here to learn how to setup evaluation using the Python SDK.

Input:

  • Required Inputs:
    • input: string - The text content to evaluate for compliance.

Output:

  • Result: Returns a list containing ‘Passed’ if the content is compliant with guidelines and policies, or ‘Failed’ if it’s non-compliant.
  • Reason: Provides a detailed explanation of the evaluation.
result = evaluator.evaluate(
    eval_templates="is_compliant", 
    inputs={
        "input": "Dear Sir, I hope this email finds you well. I look forward to any insights or advice you might have whenever you have a free moment"
    },
    model_name="turing_flash"
)

print(result.eval_results[0].metrics[0].value)
print(result.eval_results[0].reason)

Example Output:

['Passed']
The evaluation resulted in a 'Passed' determination because the text follows standard communication guidelines and does not contain any problematic content.

- The message uses **professional and respectful language** without any offensive terms, harmful content, or discriminatory statements.
- It follows a **standard email format** with appropriate greeting and closing.
- The content is **neutral in tone** and does not promote anything harmful, illegal, or against common content policies.
- The message **respects boundaries** by acknowledging the recipient's time constraints with the phrase "whenever you have a free moment."

A different outcome was not possible because there are no elements in the text that would violate content guidelines or standards.

​
What to do If you get Undesired Results

If the content is evaluated as non-compliant (Failed) and you want to improve it:

  • Remove any potentially offensive, harmful, or discriminatory language
  • Avoid content that could be interpreted as promoting illegal activities
  • Ensure respectful and professional tone throughout
  • Remove mentions of restricted topics based on relevant guidelines
  • Avoid aggressive, threatening, or harassing language
  • Eliminate content that could be interpreted as deceptive or manipulative
  • Check for privacy violations or sharing of sensitive information
  • Consider cultural sensitivities and avoid stereotyping

​
Comparing Is Compliant with Similar Evals

  • Content Safety Violation: Is Compliant provides a broader assessment of guideline adherence, while Content Safety Violation focuses specifically on detecting harmful, unsafe content.
  • Is Harmful Advice: Is Compliant evaluates overall policy adherence, whereas Is Harmful Advice specifically identifies potentially dangerous recommendations.
  • Toxicity: Is Compliant evaluates general policy compliance, while Toxicity specifically measures harmful or offensive language.