Meta-Prompt

When to use Meta-Prompt

Meta-Prompt has a teacher model analyze each round’s failures and rewrite the whole prompt, rather than patching individual parts of it. Use it when a prompt needs rethinking rather than incremental tuning.

Parameters

The On-screen label column maps the SDK parameter to the platform UI’s field label; parameters without one aren’t exposed there. The Required column reflects the SDK call signature only, not the platform form’s own required fields. The Default column is scoped the same way: in the UI, Number of Rounds is required and not prefilled, since the form starts with an empty configuration, so 5 is the SDK/backend fallback that only applies when num_rounds is left out of optimize().

Parameter	Set in	Required (SDK)	On-screen label	Default	Description
teacher_generator	MetaPromptOptimizer()	Yes	-	-	The LiteLLMGenerator that analyzes each round’s failures and rewrites the prompt
task_description	optimize()	No	Optimization Objective	"I want to improve my prompt."	What the optimized prompt should achieve
num_rounds	optimize()	No	Number of Rounds	Required in the UI; 5 in the SDK	Number of analysis-and-rewrite iterations the teacher model runs
eval_subset_size	optimize()	No	-	40	Number of dataset rows sampled for evaluation each round (capped to the dataset size)
initial_prompts	optimize()	Yes	-	-	The first prompt in initial_prompts to optimize

The teacher receives the meta-prompt, built from the current prompt, the task description, and the round’s failures, and rewrites the prompt in response. In round 1, the current prompt is the first prompt in initial_prompts; from round 2 on, it’s the teacher’s own last rewrite, alongside the earlier attempts that already scored worse.

The teacher is a LiteLLMGenerator. Weigh a stronger model against a cheaper one the same way you would for the evaluator: better rewrites versus lower per-round cost.

The example below uses gpt-4o-mini.

task_description is not specific to Meta-Prompt: every optimizer accepts it alongside its own parameters; for Meta-Prompt, it’s the goal statement the teacher rewrites the prompt against. The SDK default above only applies if you omit the argument to optimize(). A run started from the platform sends a request that must carry both task_description and num_rounds keys.

optimize() also takes evaluator, data_mapper, and dataset, shared by every optimizer’s optimize() call and covered in the SDK reference.

Raising num_rounds gives the teacher model more analyze-and-rewrite cycles before settling, at the cost of one teacher-model call per extra round, plus one generator call and one evaluator call for each row in that round’s eval subset (min(len(dataset), eval_subset_size) rows, so up to 40 by default). Start at the default of 5 and raise it if the score is still improving by the last round; lower it for a quick check.

Usage

Meta-Prompt is available from the platform UI as well as the Python SDK below.

pip install agent-opt

This installs the fi.opt namespace used in the imports below. Get fi_api_key and fi_secret_key from Admin Settings (or set FI_API_KEY/FI_SECRET_KEY as environment variables and drop them from the Evaluator call below).

from fi.opt.optimizers import MetaPromptOptimizer
from fi.opt.generators import LiteLLMGenerator
from fi.opt.datamappers import BasicDataMapper
from fi.opt.base.evaluator import Evaluator

# Dataset: a list of dicts, one per example. Keys must cover whatever the
# prompt template and key_map below reference, here just "article".
# See "Build the dataset" in /docs/optimization/guides/optimize-from-the-sdk.
my_dataset = [
    {"article": "The James Webb Space Telescope has captured its clearest images yet of a distant exoplanet's atmosphere, revealing traces of carbon dioxide and methane."},
    {"article": "Researchers have discovered a new enzyme that breaks down PET plastic at room temperature, far faster than any previously known enzyme."},
]

# Teacher model that analyzes failures and rewrites the prompt. Its
# prompt_template must be the passthrough "{prompt}": the optimizer sends
# the whole meta-prompt through the "prompt" key, not the dataset's own keys.
teacher_generator = LiteLLMGenerator(
    model="gpt-4o-mini",
    prompt_template="{prompt}"
)

# Evaluator that scores each rewrite
# eval_template and eval_model_name options: /docs/evaluation/builtin and /docs/evaluation/concepts/evaluator-models
evaluator = Evaluator(
    eval_template="summary_quality",
    eval_model_name="turing_flash",
    fi_api_key="your_key",
    fi_secret_key="your_secret"
)

# Maps generator output and dataset fields to what the evaluator expects
data_mapper = BasicDataMapper(
    key_map={"input": "article", "output": "generated_output"}
)

optimizer = MetaPromptOptimizer(teacher_generator=teacher_generator)

result = optimizer.optimize(
    initial_prompts=["Summarize this article: {article}"],
    task_description="Create concise, informative summaries",
    num_rounds=5,
    evaluator=evaluator,
    data_mapper=data_mapper,
    dataset=my_dataset
)

# Read the optimized prompt and its score off the result
print(f"Final score: {result.final_score:.4f}")
print(f"Best prompt:\n{result.best_generator.get_prompt_template()}")

A successful run prints the final score followed by the rewritten prompt, as in the two print calls above. result carries other fields beyond final_score and best_generator; see the SDK reference for the full list. If it errors instead, check that key_map in data_mapper matches both your dataset’s field names and the evaluator’s expected keys, that eval_template is a valid template name, and that your API credentials are correct.

Keep exploring