Using the Python SDK

Install the library and set environment variables so the evaluator can call Future AGI (and so your generator can call your LLM provider if needed).

pip install agent-opt

export FI_API_KEY="your_api_key"
export FI_SECRET_KEY="your_secret_key"

You can also pass fi_api_key and fi_secret_key into the Evaluator instead of using env vars.

Build a list of dicts. Each dict is one example; keys should match what your prompt and DataMapper use (e.g. article, target_summary for summarization).

dataset = [
    {"article": "The James Webb Space Telescope has captured...", "target_summary": "The JWST has taken new pictures."},
    {"article": "Researchers have discovered a new enzyme...", "target_summary": "A new enzyme that rapidly breaks down plastics has been found."},
    # ... more rows
]

The Evaluator scores each prompt’s outputs. The DataMapper maps your dataset keys to the eval’s expected keys (input, output, etc.).

from fi.opt.base.evaluator import Evaluator
from fi.opt.datamappers import BasicDataMapper

evaluator = Evaluator(
    eval_template="summary_quality",
    eval_model_name="turing_flash",
    fi_api_key="your_key",      # or rely on env FI_API_KEY
    fi_secret_key="your_secret" # or rely on env FI_SECRET_KEY
)

data_mapper = BasicDataMapper(
    key_map={"input": "article", "output": "generated_output"}
)

Pick an optimizer that fits your task (e.g. Random Search for a quick baseline, Meta-Prompt for deep refinement, GEPA for production-grade results). Some optimizers need a generator or teacher model; others take model names and config.

Example: Random Search (simple baseline)

from fi.opt.optimizers import RandomSearchOptimizer
from fi.opt.generators import LiteLLMGenerator

initial_generator = LiteLLMGenerator(
    model="gpt-4o-mini",
    prompt_template="Summarize this: {article}"
)

optimizer = RandomSearchOptimizer(
    generator=initial_generator,
    teacher_model="gpt-4o",
    num_variations=5
)

Example: Meta-Prompt (deep reasoning)

from fi.opt.optimizers import MetaPromptOptimizer
from fi.opt.generators import LiteLLMGenerator

teacher = LiteLLMGenerator(model="gpt-4o", prompt_template="{prompt}")
optimizer = MetaPromptOptimizer(teacher_generator=teacher, num_rounds=5)

For a detailed comparison, see the Optimizers overview.

Call optimizer.optimize() with the evaluator, data mapper, dataset, and any optimizer-specific options (e.g. initial_prompts, eval_subset_size, task_description).

rinitial_prompt = "Summarize the following article: {article}"

result = optimizer.optimize(
    evaluator=evaluator,
    data_mapper=data_mapper,
    dataset=dataset,
    initial_prompts=[initial_prompt],
    task_description="Generate a concise, one-sentence summary of the article.",
    eval_subset_size=10  # Use a subset of the data for faster evaluation per round
)

Use the returned result object: result.final_score, result.best_generator.get_prompt_template(), and result.history for each round or variation.

# Print the final score and the best prompt found
print(f"Final Score: {result.final_score:.4f}")
print(f"Best Prompt:\n{result.best_generator.get_prompt_template()}")

# Review the history of the optimization
for i, iteration in enumerate(result.history):
    print(f"\n--- Round {i+1} ---")
    print(f"Score: {iteration.average_score:.4f}")
    print(f"Prompt: {iteration.prompt}")