TTS Accuracy

Evaluates the accuracy and naturalness of text-to-speech output, including pronunciation, emphasis, and emotional tone.

TTS Accuracy checks whether a text-to-speech output faithfully conveys the intended text, in wording, pronunciation, emphasis, and emotional tone. Run it to catch TTS engines that drift from the source text or sound unnatural.

What it does

TTS Accuracy is an LLM-as-Judge eval. It reads the source text and listens to the generated audio, then scores how accurately the audio reflects the intended message.

Input

Required Input	Type	Description
`text`	`string`	The original text input that was converted to speech
`generated_audio`	`string`	URL or file path to the TTS audio output to be evaluated

Output

Field	Type	Description
Result	`score`	Higher scores indicate more accurate TTS output
Reason	`string`	A plain-language explanation of the TTS accuracy assessment

Run it from code

Call evaluate() with the template name and the eval’s required inputs. It returns the score and the reason.

Note

Before running: install the SDK and set FI_API_KEY / FI_SECRET_KEY. The model argument in the snippets is the evaluator model Future AGI uses to run the eval; turing_flash is a fast default.

from fi.evals import evaluate

result = evaluate(
    "TTS_accuracy",
    text="Welcome to our service. How can I help you today?",
    generated_audio="https://example.com/tts-output.wav",
    model="turing_flash",
)

print(result.score)
print(result.reason)

import { evaluate } from "@future-agi/ai-evaluation";

const result = await evaluate(
  "TTS_accuracy",
  {
    text: "Welcome to our service. How can I help you today?",
    generated_audio: "https://example.com/tts-output.wav"
  },
  { modelName: "turing_flash" }
);

console.log(result);

When to use

Run TTS Accuracy wherever text is converted to speech and the output needs to represent that text faithfully.

Text-to-speech pipelines, to confirm generated audio matches the source text
Conversational voice agents, where mispronunciation or wrong emphasis changes meaning
Audio outputs where emotional tone needs to match the context of the text

What to do when TTS Accuracy fails

Check for mispronounced words, especially proper nouns, technical terms, or abbreviations
Verify that emphasis and stress are placed on the correct syllables
Review the emotional tone: it should match the context of the text
Ensure the audio is clear and free from artifacts or distortion
Consider using phonetic spelling or SSML tags to guide the TTS engine

Was this page helpful?

Questions & Discussion