Optimization

What optimization is

Optimization rewrites your agent’s prompt automatically, scored by the same evals your simulation runs. Rather than hand-editing the prompt and rerunning, you let an algorithm propose many candidate prompts, score each one, and hand you the best. The prompt is what a run changes, few-shot examples included; your agent definition and your evals stay as they are.

Reach for it when hand-fixing has stalled. Fix My Agent is the lighter first move: it reads a finished run and hands you a prioritised list of issues to fix yourself. An optimization run goes further and does the rewriting for you.

How an optimization run works

An optimization run starts from a simulation run you have already completed. It samples the conversations recorded in that run, and every candidate prompt is scored against that same sample, so the comparison holds still while the search moves.

%%{init: {"flowchart": {"curve": "basis", "rankSpacing": 75, "nodeSpacing": 55, "padding": 20}}}%%
flowchart TB
  RUN["Simulation run<br/>conversations + eval scores"] -->|"frozen sample"| SCORE
  subgraph SEARCH["The search loop"]
      direction LR
      ALG["Algorithm"] -->|"proposes"| CAND["Candidate prompt"]
      CAND --> SCORE["Trial score"]
      SCORE -->|"guides the next round"| ALG
  end
  SEARCH --> BEST["Best prompt"]

Say your refund agent keeps failing a resolution eval. You point an optimization run at the simulation run where it failed, the algorithm generates candidate prompts, each is scored on that same eval against those conversations, and the best-performing prompt surfaces for you to review and apply. Because the score is your own eval, the winner is the prompt that best satisfies the bar you set.

The algorithms

You pick the search strategy. They differ in how hard they search and in what they change, and searching harder costs more model calls. Start with Random Search for a baseline, then match the pick to what’s wrong:

• Random Search tries simple variations, the cheapest way to see how much room a prompt has
• Bayesian keeps your wording and searches over which few-shot examples and settings work best, so reach for it when the prompt reads fine but the examples feel arbitrary
• ProTeGi critiques each failure and applies a targeted fix, keeping several candidate revisions in play at once, for a prompt that is mostly right
• Meta-Prompt analyses failures and rewrites the whole prompt through deeper reasoning, for a prompt that needs rethinking rather than patching
• PromptWizard mutates the prompt across different thinking styles, then critiques and refines the top performers
• GEPA runs an evolutionary search across generations of candidates, the widest search of the six

The Optimization product docs cover each algorithm in depth.

Keep exploring