Translation Accuracy

Evaluates the quality of translation by checking semantic accuracy, cultural appropriateness, and preservation of original meaning. It considers both literal accuracy and natural expression in the target language. 

result = evaluator.evaluate(
    eval_templates="translation_accuracy",
    inputs={
        "input": "Hello, how are you?",
        "output": "¡Hola, cómo estás?"
    },
    model_name="turing_flash"
)

print(result.eval_results[0].output)
print(result.eval_results[0].reason)
import { Evaluator, Templates } from "@future-agi/ai-evaluation";

const evaluator = new Evaluator();

const result = await evaluator.evaluate(
  "translation_accuracy",
  {
    input: "Hello, how are you?",
    output: "¡Hola, cómo estás?"
  },
  {
    modelName: "turing_flash",
  }
);

console.log(result);
Input
Required InputTypeDescription
inputstringContent in source language.
outputstringContent in translated language.
Output
FieldDescription
ResultReturns a score representing the translation accuracy, where higher values indicate superior translation quality.
ReasonProvides a detailed explanation of the translation accuracy assessment.

What to Do When Translation Accuracy Evaluation Gives a Low Score

Reassess the evaluation criteria to ensure they are well-defined and aligned with the evaluation’s objectives, making adjustments if necessary to enhance their comprehensiveness and relevance. Analyse the translation for semantic accuracy, cultural appropriateness, and natural linguistic expression, identifying any discrepancies that may affect meaning. If inconsistencies are found, refine the translation to ensure it accurately conveys the original intent while maintaining contextual and cultural integrity.

Comparing Translation Accuracy with Similar Evals

  • Groundedness: Translation Accuracy ensures meaning is accurately conveyed across languages, while Groundedness ensures the response strictly adheres to provided context without adding external information.
  • Fuzzy Match: Translation Accuracy evaluates quality, accuracy, and cultural appropriateness of translations, while Fuzzy Match compares texts for approximate similarity using surface-level matching.
Was this page helpful?

Questions & Discussion