Synthetic Image Evaluator
Evaluates whether an image was generated by AI or captured by a camera, scoring confidence that it's synthetic.
Synthetic Image Evaluator checks whether an image was created by an AI generation model or captured by a camera. Run it wherever you need to tell synthetic images apart from real ones.
What it does
Synthetic Image Evaluator is an LLM-as-Judge eval. It reads an image and scores how confident it is that the image is AI-generated rather than a real photograph.
Input
| Required Input | Type | Description |
|---|---|---|
image | string | URL or file path to the image to be evaluated |
Output
| Field | Type | Description |
|---|---|---|
| Result | score | Higher values indicate greater confidence that the image is AI-generated |
| Reason | string | A plain-language explanation of why the image was classified as AI-generated or not |
Run it from code
Call evaluate() with the template name and the eval’s required inputs. It returns the score and the reason.
Note
Before running: install the SDK and set FI_API_KEY / FI_SECRET_KEY. The model argument in the snippets is the evaluator model Future AGI uses to run the eval; turing_flash is a fast default.
from fi.evals import evaluate
result = evaluate(
"synthetic_image_evaluator",
image="https://www.esparklearning.com/app/uploads/2024/04/Albert-Einstein-generated-by-AI-1024x683.webp",
model="turing_flash",
)
print(result.score)
print(result.reason)import { evaluate } from "@future-agi/ai-evaluation";
const result = await evaluate(
"synthetic_image_evaluator",
{
image: "https://www.esparklearning.com/app/uploads/2024/04/Albert-Einstein-generated-by-AI-1024x683.webp"
},
{ modelName: "turing_flash" }
);
console.log(result);When to use
Run Synthetic Image Evaluator wherever you need to verify the provenance of an image before trusting or publishing it.
- Image pipelines where you need to confirm whether an input image is a real photograph or AI-generated
- Content moderation workflows that treat synthetic and real images differently
- Auditing datasets that mix real and generated images
What to do when Synthetic Image Evaluator fails
For actual photographs mistakenly identified as synthetic, ensure the image hasn’t been heavily processed or filtered, check that it doesn’t have unusual artifacts from compression or editing, and consider providing a higher resolution version if available.
For synthetic images that aren’t being detected, keep in mind that newer AI generation models are increasingly photorealistic, and images that were post-processed or combined with real photographs can be harder to detect. The evaluation works best with full images rather than small crops or heavily modified versions.
Questions & Discussion