Evaluate Translation

Accurate translation is essential for ensuring that content is linguistically precise, contextually appropriate, and culturally relevant across different languages. High-quality translations must preserve the original meaning, intent, and tone, while also adhering to the grammatical and stylistic conventions of the target language.

Challenges associated with poor translation quality include:

• Loss of Meaning – Misinterpretation or distortion of the original message.
• Cultural Insensitivity – Use of phrases or expressions that do not align with cultural norms.
• Unnatural Language – Translations that are overly literal, mechanical, or awkwardly phrased.

To assess translation effectiveness, Translation Accuracy Evaluation measures how well a translation conveys the original meaning while ensuring linguistic fluency and cultural appropriateness.

Click here to read the eval definition of Translation Accuracy

​

a. Using Interface

​

Required Inputs

• input: The original text.
• output: The translated text.

​

Configuration Parameters

• Check Internet: Whether to verify information using external sources for contextual validation.

​

Output

• Returns a score between 0 and 1, where higher values indicate superior translation quality.

​

b. Using SDK

from fi.testcases import TestCase
from fi.evals.templates import TranslationAccuracy

test_case = TestCase(
    input="Hello, how are you?",
    output="¡Hola, cómo estás?",
)

template = TranslationAccuracy()

response = evaluator.evaluate(eval_templates=[template], inputs=[test_case], model_name="turing_flash")

print(f"Score: {response.eval_results[0].metrics[0].value}")
print(f"Reason: {response.eval_results[0].reason}")