Accurate translation is essential for ensuring that content is linguistically precise, contextually appropriate, and culturally relevant across different languages. High-quality translations must preserve the original meaning, intent, and tone, while also adhering to the grammatical and stylistic conventions of the target language.

Challenges associated with poor translation quality include:

  • Loss of Meaning – Misinterpretation or distortion of the original message.
  • Cultural Insensitivity – Use of phrases or expressions that do not align with cultural norms.
  • Unnatural Language – Translations that are overly literal, mechanical, or awkwardly phrased.

To assess translation effectiveness, Translation Accuracy Evaluation measures how well a translation conveys the original meaning while ensuring linguistic fluency and cultural appropriateness.

Click here to read the eval definition of Translation Accuracy


a. Using Interface

Required Inputs

  • input: The original text.
  • output: The translated text.

Configuration Parameters

  • Check Internet: Whether to verify information using external sources for contextual validation.

Output

  • Returns a score between 0 and 1, where higher values indicate superior translation quality.

b. Using SDK

Export your API key and Secret key into your environment variables.
from fi.testcases import TestCase
from fi.evals.templates import TranslationAccuracy

test_case = TestCase(
    input="Hello, how are you?",
    output="¡Hola, cómo estás?",
)

template = TranslationAccuracy()

response = evaluator.evaluate(eval_templates=[template], inputs=[test_case], model_name="turing_flash")

print(f"Score: {response.eval_results[0].metrics[0].value}")
print(f"Reason: {response.eval_results[0].reason}")