Optimize Your First Prompt

Install the library and set environment variables so the optimizer can call Future AGI for evaluations. Get your API keys from the Future AGI dashboard.

pip install agent-opt

import os

os.environ["FI_API_KEY"] = "YOUR_API_KEY"
os.environ["FI_SECRET_KEY"] = "YOUR_SECRET_KEY"

Optimization is data-driven. Use a list of Python dicts: each item is one example (e.g. input text and target or reference). Here, a small summarization dataset:

dataset = [
    {
        "article": "The James Webb Space Telescope has captured stunning new images of the Pillars of Creation, revealing intricate details of gas and dust clouds where new stars are forming.",
        "target_summary": "The JWST has taken new, detailed pictures of the Pillars of Creation."
    },
    {
        "article": "Researchers have discovered a new enzyme that can break down plastics at record speed, offering a potential solution to the global plastic pollution crisis.",
        "target_summary": "A new enzyme that rapidly breaks down plastics has been found."
    },
]

Set up the generator (initial prompt + model), evaluator (eval template + model), data mapper (dataset keys → eval inputs), and the optimizer. Then call optimizer.optimize().

from fi.opt.optimizers import RandomSearchOptimizer
from fi.opt.generators import LiteLLMGenerator
from fi.opt.datamappers import BasicDataMapper
from fi.opt.base.evaluator import Evaluator

# a. Define the generator with the initial prompt to be optimized
initial_generator = LiteLLMGenerator(
    model="gpt-4o-mini",
    prompt_template="Summarize this: {article}"
)

# b. Setup the evaluator to score prompt performance
evaluator = Evaluator(
    eval_template="summary_quality",  # A built-in template for summarization
    eval_model_name="turing_flash"    # The model to perform the evaluation
)

# c. Setup the data mapper to link dataset fields
data_mapper = BasicDataMapper(
    key_map={"input": "article", "output": "generated_output"}
)

# d. Initialize the Random Search optimizer
optimizer = RandomSearchOptimizer(
    generator=initial_generator,
    teacher_model="gpt-4o",  # A powerful model to generate prompt ideas
    num_variations=5         # Generate 5 different versions of our prompt
)

# e. Run the optimization!
result = optimizer.optimize(
    evaluator=evaluator,
    data_mapper=data_mapper,
    dataset=dataset
)

The result object holds the best prompt and score, plus the history of tried variations.

# Print the best prompt and its score
print(f"--- Optimization Complete ---")
print(f"Final Score: {result.final_score:.4f}")
print(f"Best Prompt Found:\n{result.best_generator.get_prompt_template()}")

# You can also review the history of all tried variations
for i, iteration in enumerate(result.history):
    print(f"\n--- Variation {i+1} ---")
    print(f"Score: {iteration.average_score:.4f}")
    print(f"Prompt: {iteration.prompt}")

After your first run you can try other optimizers, tune num_variations or eval templates, or use the same pattern with your own dataset and prompt. See the links below for deeper guides.