GEPA Optimization

GEPA needs a dataset of inputs to score candidate prompts against. Each row is a plain dict.

dataset = [
    {
        "article": "The James Webb Space Telescope has captured stunning new images of the Pillars of Creation, revealing intricate details of gas and dust clouds where new stars are forming.",
    },
    {
        "article": "Researchers at the University of Austin have discovered a new enzyme capable of breaking down PET plastic, the material commonly found in beverage bottles, in a matter of hours.",
    },
]

initial_prompt = "Summarize this article concisely: {article}"

You should see nothing yet, this just defines the dataset and starting prompt in memory. Two rows is enough to run the tutorial; use 30+ for a result you’d trust.

The Evaluator scores each candidate prompt’s output using a Future AGI eval template.

from fi.opt.base import Evaluator

evaluator = Evaluator(
    eval_template="summary_quality",
    eval_model_name="turing_flash",
)

You should see nothing yet, this just constructs the evaluator. If FI_API_KEY or FI_SECRET_KEY isn’t set, construction raises ValueError naming the missing key.

BasicDataMapper tells the optimizer which dataset column is the input and which generated field is the output to score.

from fi.opt.datamappers import BasicDataMapper

data_mapper = BasicDataMapper(
    key_map={"input": "article", "output": "generated_output"}
)

generated_output is the field name the generator writes to during a run, not a column in dataset.

You should see nothing yet, this just builds the mapper; it does no work until optimize() runs.

GEPA needs two models: a reflection model that critiques failures and rewrites the prompt, and a generator model whose prompt is being optimized.

from fi.opt.optimizers import GEPAOptimizer

optimizer = GEPAOptimizer(
    reflection_model="gpt-4o",       # critiques failures and proposes mutations
    generator_model="gpt-4o-mini",   # the "student" model whose prompt is optimized
)

A stronger reflection model produces more useful critiques and better mutations. It’s worth spending more here than on the generator model.

You should see nothing yet, this just constructs the optimizer; no API calls happen until optimize() runs.

max_metric_calls is the total evaluation budget for the whole run, including the calls spent scoring the initial prompt.

result = optimizer.optimize(
    evaluator=evaluator,
    data_mapper=data_mapper,
    dataset=dataset,
    initial_prompts=[initial_prompt],
    max_metric_calls=20,
)

This calls the reflection model to analyze failures, mutates the prompt, and scores each new candidate with the evaluator. On a 2-row dataset with a small budget it typically finishes in under a minute; the default budget of 150 takes longer.

seed_score = result.history[0].average_score

print(f"Seed Score: {seed_score:.4f}")
print(f"Best Score: {result.final_score:.4f}")
print(f"Initial Prompt:\n{initial_prompt}")
print(f"Best Prompt Found:\n{result.best_generator.get_prompt_template()}")

result.history[0] is the seed candidate’s evaluation, appended before any mutation, so seed_score and result.final_score give you a before/after delta rather than a single number. Example output, illustrative:

Seed Score: 0.7200
Best Score: 0.8400
Initial Prompt:
Summarize this article concisely: {article}
Best Prompt Found:
Summarize this article in 1-2 sentences, naming the specific finding or event and its concrete outcome...

The exact prompt and score depend on your dataset and models, so don’t expect to match this run.