FID Score

Computes the Frechet Inception Distance between two sets of images, where lower scores indicate more similar distributions.

FID Score measures how close the distribution of generated images is to a set of real reference images. Run it to track the statistical quality of an image generation model across a batch, not just one image at a time.

What it does

FID Score is a statistical metric. It compares the distribution of a real image set against a generated image set and scores how similar they are.

Input

Required Input	Type	Description
`real_images`	`list[string]`	List of URLs or file paths to the real/reference images
`fake_images`	`list[string]`	List of URLs or file paths to the generated/fake images

Output

Field	Type	Description
Result	`score`	Lower values indicate more similar distributions between real and generated images
Reason	`string`	A plain-language explanation of the FID score assessment

Run it from code

Call evaluate() with the template name and the eval’s required inputs. It returns the score and the reason.

Note

Before running: install the SDK and set FI_API_KEY / FI_SECRET_KEY. The model argument in the snippets is the evaluator model Future AGI uses to run the eval; turing_flash is a fast default.

from fi.evals import evaluate

result = evaluate(
    "fid_score",
    real_images=["https://example.com/real1.jpg", "https://example.com/real2.jpg"],
    fake_images=["https://example.com/generated1.jpg", "https://example.com/generated2.jpg"],
    model="turing_flash",
)

print(result.score)
print(result.reason)

import { evaluate } from "@future-agi/ai-evaluation";

const result = await evaluate(
  "fid_score",
  {
    real_images: ["https://example.com/real1.jpg", "https://example.com/real2.jpg"],
    fake_images: ["https://example.com/generated1.jpg", "https://example.com/generated2.jpg"]
  },
  { modelName: "turing_flash" }
);

console.log(result);

When to use

Run FID Score wherever you need a distributional check on a batch of generated images, not a single image at a time.

Evaluating an image generation model’s overall output quality against a reference set
Comparing checkpoints during model training or fine-tuning
Regression testing a generation pipeline after changes to prompts, sampling, or model version

What to do when FID Score is high

Increase the diversity and size of both image sets for a more reliable score, and review the generation model for mode collapse or quality issues. Ensure real and generated images are from the same domain and resolution.

Check preprocessing steps, since both sets should be normalized consistently, and consider fine-tuning the generation model on domain-specific data.

Was this page helpful?

Questions & Discussion