Translation Accuracy

Evaluates translation quality by checking semantic accuracy, cultural appropriateness, and preserved meaning.

Translation Accuracy checks whether a translation preserves the source’s meaning, tone, and cultural context, not just its words. Run it wherever a model translates content between languages.

What it does

Translation Accuracy is an LLM-as-Judge eval. It reads the source content and the translated output, then scores the translation’s semantic accuracy and cultural appropriateness.

Input

Required Input	Type	Description
`input`	`string`	Content in source language
`output`	`string`	Content in translated language

Output

Field	Type	Description
Result	`score`	Higher values indicate superior translation quality
Reason	`string`	A plain-language explanation of the translation accuracy assessment

Run it from code

Call evaluate() with the template name and the eval’s required inputs. It returns the score and the reason.

Note

Before running: install the SDK and set FI_API_KEY / FI_SECRET_KEY. The model argument in the snippets is the evaluator model Future AGI uses to run the eval; turing_flash is a fast default.

from fi.evals import evaluate

result = evaluate(
    "translation_accuracy",
    input="Hello, how are you?",
    output="¡Hola, cómo estás?",
    model="turing_flash",
)

print(result.score)
print(result.reason)

import { evaluate } from "@future-agi/ai-evaluation";

const result = await evaluate(
  "translation_accuracy",
  {
    input: "Hello, how are you?",
    output: "¡Hola, cómo estás?"
  },
  { modelName: "turing_flash" }
);

console.log(result);

When to use

Run Translation Accuracy wherever content moves from one language to another and meaning needs to survive the transfer.

Text and audio content translated into a target language
RAG and retrieval pipelines, where translated content feeds downstream retrieval or generation
Localization checks, to confirm cultural appropriateness alongside literal correctness

What to do when Translation Accuracy fails

Reassess the evaluation criteria to ensure they’re well-defined and aligned with the evaluation’s objectives, making adjustments if necessary to enhance their comprehensiveness and relevance.

Analyze the translation for semantic accuracy, cultural appropriateness, and natural linguistic expression, identifying any discrepancies that may affect meaning. If inconsistencies are found, refine the translation to accurately convey the original intent while maintaining contextual and cultural integrity.

Was this page helpful?

Questions & Discussion