Skip to main content

​
Evaluation Using Interface

Input:
  • Required Inputs:
    • input_audio: The audio file (URL or local path) to be evaluated.
Output:
  • Score: A percentage score between 0 and 100, where 100 represents perfect audio quality.
  • Reason: A detailed explanation of the quality assessment in markdown bullet point format.

​
Evaluation Using SDK

Click here to learn how to setup evaluation using SDK.
Input:
  • Required Inputs:
    • input_audio: string - The file path or URL to the audio file to be evaluated.
Output:
  • Score: Returns a float value between 0 and 1, where higher values indicate better audio quality.
  • Reason: Provides a detailed explanation of the audio quality assessment.
result = evaluator.evaluate(
    eval_templates="audio_quality",
    inputs={
        "input_audio": "https://datasets-server.huggingface.co/assets/EarthSpeciesProject/NatureLM-audio-training/--/e98500754629b63dd8d2400c1a20798337da92f5/--/NatureLM-audio-training/train/0/audio/audio.wav"
    },
    model_name="turing_flash"
)

print(result.eval_results[0].output)
print(result.eval_results[0].reason)
Example Output: 
1.0
Strictly evidence-based explanation of the determination, explanation should be in markdown format with bullet points only
- The audio is a technically flawless field recording, exhibiting exceptional clarity and fidelity.
- There is a complete absence of any technical artifacts such as microphone hiss, electrical hum, clipping, or compression noise. The signal-to-noise ratio is outstanding.
- Each element of the soundscape, from the prominent bird calls in the foreground to the more distant ambient sounds, is rendered with pristine sharpness and detail.
- The recording achieves a perfect sense of naturalness and immersion, effectively transporting the listener to the environment. This represents the highest level of quality for this type of audio content.
- The audio quality aligns perfectly with the definition of a 1.0 score: "exceptional, exhibiting flawless clarity" and representing "professional excellence."

​
What to do If you get Undesired Results

If the audio quality score is lower than expected:
  • Check for background noise or interference in the recording
  • Verify the recording environment is appropriate (e.g., proper acoustic treatment)
  • Ensure the microphone or recording device is of sufficient quality
  • Consider using noise reduction techniques in post-processing
  • Check for issues like clipping, distortion, or compression artifacts
  • Verify the audio file format and bitrate are appropriate for the intended use
  • Re-record in a more controlled environment if possible

​
Comparing Audio Quality with Similar Evals

  • Audio Transcription: While Audio Quality evaluates the perceptual quality of the audio itself, Audio Transcription assesses the accuracy of converting speech in the audio to text.