Accurate translation is essential for ensuring that content is linguistically precise, contextually appropriate, and culturally relevant across different languages. High-quality translations must preserve the original meaning, intent, and tone, while also adhering to the grammatical and stylistic conventions of the target language.

Challenges associated with poor translation quality include:

  • Loss of Meaning – Misinterpretation or distortion of the original message.
  • Cultural Insensitivity – Use of phrases or expressions that do not align with cultural norms.
  • Unnatural Language – Translations that are overly literal, mechanical, or awkwardly phrased.

To assess translation effectiveness, Translation Accuracy Evaluation measures how well a translation conveys the original meaning while ensuring linguistic fluency and cultural appropriateness.

Click here to read the eval definition of Translation Accuracy

​
a. Using Interface

​
Required Inputs

  • input: The original text.
  • output: The translated text.

​
Configuration Parameters

  • Check Internet: Whether to verify information using external sources for contextual validation.

​
Output

  • Returns a score between 0 and 1, where higher values indicate superior translation quality.

​
b. Using SDK

from fi.testcases import TestCase
from fi.evals.templates import TranslationAccuracy

test_case = TestCase(
    input="Hello, how are you?",
    output="¡Hola, cómo estás?",
)

template = TranslationAccuracy(config={
    "check_internet": False
})

response = evaluator.evaluate(eval_templates=[template], inputs=[test_case])

print(f"Score: {response.eval_results[0].metrics[0].value}")
print(f"Reason: {response.eval_results[0].reason}")
Detect Ambiguous Prompts Using Prompt Perplexity EvalEvaluate Summary