Audio Quality

Evaluates the perceptual quality of an audio input, assessing aspects like clarity, noise levels, and overall listenability using an LLM. 

result = evaluator.evaluate(
    eval_templates="audio_quality",
    inputs={
        "input_audio": "https://datasets-server.huggingface.co/assets/EarthSpeciesProject/NatureLM-audio-training/--/e98500754629b63dd8d2400c1a20798337da92f5/--/NatureLM-audio-training/train/0/audio/audio.wav"
    },
    model_name="turing_flash"
)

print(result.eval_results[0].output)
print(result.eval_results[0].reason)
import { Evaluator, Templates } from "@future-agi/ai-evaluation";

const evaluator = new Evaluator();

const result = await evaluator.evaluate(
  "audio_quality",
  {
    input_audio: "https://datasets-server.huggingface.co/assets/EarthSpeciesProject/NatureLM-audio-training/--/e98500754629b63dd8d2400c1a20798337da92f5/--/NatureLM-audio-training/train/0/audio/audio.wav"
  },
  {
    modelName: "turing_flash",
  }
);

console.log(result);
Input
Required InputTypeDescription
input_audiostringThe file path or URL to the audio file to be evaluated
Output
FieldDescription
ResultReturns a numeric score where higher score indicates better audio quality
ReasonProvides a detailed explanation of the audio quality assessment

What to do If you get Undesired Results

If the audio quality score is lower than expected:

  • Check for background noise or interference in the recording
  • Verify the recording environment is appropriate (e.g., proper acoustic treatment)
  • Ensure the microphone or recording device is of sufficient quality
  • Consider using noise reduction techniques in post-processing
  • Check for issues like clipping, distortion, or compression artifacts
  • Verify the audio file format and bitrate are appropriate for the intended use
  • Re-record in a more controlled environment if possible

Comparing Audio Quality with Similar Evals

  • Audio Transcription: While Audio Quality evaluates the perceptual quality of the audio itself, Audio Transcription assesses the accuracy of converting speech in the audio to text.
Was this page helpful?

Questions & Discussion