Optimization fundamentals

What it is

Experimentation allows users to compare different prompt or model configurations, but it does not refine a single prompt in a systematic, data-driven way. Once an experiment identifies a well-performing prompt, optimization takes it a step further by making iterative improvements. This process enhances clarity, response quality, and efficiency while reducing ambiguity that can cause inconsistencies in AI outputs.

Since LLMs generate responses probabilistically, even the same input can produce different outputs. Optimization ensures that prompts are structured to deliver the most consistent, high-quality results while minimising unnecessary token usage.

---

Why optimization

Experimentation compares prompt or model configurations but doesn’t refine a single prompt systematically. Optimization does: it uses eval scores to guide changes so you get clearer, more consistent, and more efficient outputs. Because LLM outputs vary, optimization helps structure prompts for the most consistent, high-quality results and fewer wasted tokens.

---

How optimization works

An optimization run needs: a dataset, an initial prompt, evaluation metrics to score performance, and an optimization algorithm. The optimizer runs the baseline, then enters a loop: it creates new prompt candidates, runs them on the dataset, scores them with your evals, and uses that feedback for the next round. When the run finishes, you get the best-performing prompt (and optionally a history of trials). You can change where optimization runs (e.g. project or dataset) in settings; that applies everywhere.

---

Choosing an optimization strategy

The library provides six algorithms. Use the table and tabs below to pick one.

Quick selection

Use case	Recommended optimizer	Why
Few-shot learning	Bayesian Search	Selects and formats examples intelligently
Complex reasoning	Meta-Prompt	Deep failure analysis and full prompt rewrite
Fixing specific errors	ProTeGi	Identifies and fixes failure patterns
Creative / open-ended	PromptWizard	Diverse prompt exploration
Production deployments	GEPA	Strong evolutionary search with good budgeting
Quick baseline	Random Search	Fast, simple baseline

Performance comparison

Optimizer	Speed	Quality	Cost	Dataset size
Bayesian Search	⚡⚡	⭐⭐⭐⭐	💰💰	15–50
Meta-Prompt	⚡⚡	⭐⭐⭐⭐	💰💰💰	20–40
ProTeGi	⚡	⭐⭐⭐⭐	💰💰💰	20–50
PromptWizard	⚡	⭐⭐⭐⭐	💰💰💰	15–40
GEPA	⚡	⭐⭐⭐⭐⭐	💰💰💰💰	30–100
Random Search	⚡⚡⚡	⭐⭐	💰	10–30

Optimizer details

Decision Tree

Do you need production-grade optimization?
├─ Yes → Use GEPA
└─ No
   │
   Do you have few-shot examples in your dataset?
   ├─ Yes → Use Bayesian Search
   └─ No
      │
      Is your task reasoning-heavy or complex?
      ├─ Yes → Use Meta-Prompt
      └─ No
         │
         Do you have clear failure patterns to fix?
         ├─ Yes → Use ProTeGi
         └─ No
            │
            Do you want creative exploration?
            ├─ Yes → Use PromptWizard
            └─ No → Use Random Search (baseline)

---

Combining Optimizers

You can run multiple optimizers sequentially for best results:

# Stage 1: Quick exploration with Random Search
random_result = random_optimizer.optimize(...)
initial_prompts = [h.prompt for h in random_result.history[:3]]

# Stage 2: Deep refinement with Meta-Prompt
meta_result = meta_optimizer.optimize(
    initial_prompts=initial_prompts,
    ...
)

# Stage 3: Few-shot enhancement with Bayesian Search
final_result = bayesian_optimizer.optimize(
    initial_prompts=[meta_result.best_generator.get_prompt_template()],
    ...
)

---

What you can do next