Image Instruction Adherence
Measures how well generated images adhere to a text instruction across subject, style, and composition.
Image Instruction Adherence checks whether a generated image actually follows the text instruction it was prompted with. Run it to catch generations that drift from the requested subject, style, or composition.
What it does
Image Instruction Adherence is an LLM-as-Judge eval. It reads the instruction and the generated image, then scores how closely the image adheres to the instruction.
Input
| Required Input | Type | Description |
|---|---|---|
instruction | string | The text instruction describing what the image should contain or depict |
images | string or list[string] | The generated image(s) to be evaluated against the instruction |
Output
| Field | Type | Description |
|---|---|---|
| Result | score | Higher values indicate closer adherence to the instruction |
| Reason | string | A plain-language explanation of how well the image matches the instruction |
Run it from code
Call evaluate() with the template name and the eval’s required inputs. It returns the score and the reason.
Note
Before running: install the SDK and set FI_API_KEY / FI_SECRET_KEY. The model argument in the snippets is the evaluator model Future AGI uses to run the eval; turing_flash is a fast default.
from fi.evals import evaluate
result = evaluate(
"image_instruction_adherence",
instruction="A photorealistic image of a red sports car on a mountain road at sunset",
images=["https://example.com/generated-car.jpg"],
model="turing_flash",
)
print(result.score)
print(result.reason)import { evaluate } from "@future-agi/ai-evaluation";
const result = await evaluate(
"image_instruction_adherence",
{
instruction: "A photorealistic image of a red sports car on a mountain road at sunset",
images: ["https://example.com/generated-car.jpg"]
},
{ modelName: "turing_flash" }
);
console.log(result);When to use
Run Image Instruction Adherence wherever an image is generated from a text instruction and you need to confirm the output matches what was asked for.
- Text-to-image generation pipelines, to verify subject, style, and composition match the prompt
- Comparing generation models or prompt variants on instruction-following quality
- Reviewing generations before they ship, when the instruction has multiple specific requirements
What to do when Image Instruction Adherence fails
Review the instruction for ambiguity and make it more specific, and check that all key elements mentioned in the instruction are present in the image. Verify that style, composition, and color requirements are reflected.
Consider iterating on the generation prompt to better guide the model, and break complex instructions into simpler, more focused prompts.
Questions & Discussion