​
Evaluation Using Interface

Input:

  • Required Inputs:
    • input: The text content column to analyse for cultural appropriateness.

Output:

  • Result: “Passed” / “Failed”

Interpretation:

  • Passed: Indicates the content is culturally appropriate and inclusive.
  • Failed: Indicates the presence of potential cultural insensitivity, bias, or exclusionary language.

​
Evaluation Using Python SDK

Click here to learn how to setup evaluation using the Python SDK.

InputParameterTypeDescription
Required InputsinputstringThe text content to analyse for cultural appropriateness.
OutputTypeDescription
ResultboolReturns 1 (indicating culturally appropriate content) or 0 (indicating potential cultural insensitivity).
from fi.testcases import TestCase
from fi.evals.templates import CulturalSensitivity

cultural_eval = CulturalSensitivity()

test_case = TestCase(
    input="This is a sample text to check for cultural sensitivity"
)

result = evaluator.evaluate(eval_templates=[cultural_eval], inputs=[test_case], model_name="turing_flash")
sensitivity_result = result.eval_results[0].metrics[0].value

​
What to Do When Cultural Sensitivity Fails

Review the evaluation criteria to ensure they are well-defined and aligned with the assessment’s objectives. If necessary, the criteria should be adjusted to ensure they comprehensively address inclusivity and cultural awareness.

Next, a detailed analysis of the text should be conducted to identify any language that may be biased, exclusionary, or insensitive. Refinements should be made to enhance cultural appropriateness, ensuring that the text respects diverse perspectives and promotes inclusivity.

​
Differentiating Cultural Sensitivity with Tone

Cultural Sensitivity focuses on ensuring that language and content are appropriate within cultural contexts, promoting inclusivity and respect for diversity. In contrast, Tone evaluation identifies the emotional tone of the text, categorising it into specific emotional states.

Cultural Sensitivity provides a Pass/Fail result based on the presence of culturally insensitive language, whereas Tone evaluation produces a set of choices indicating the emotional tone detected.