Read optimization results

Open a run from its dataset’s Optimization tab once it’s finished, and you’ll land on the detail page this guide walks through. (Haven’t started a run yet? See Run an optimization.)

That page is where you find out whether the optimizer actually improved anything, both overall and on the individual evals you selected.

The run detail page

Once a run is Completed, the detail page shows a score graph, a result bar sitting between the graph and the list with the improvement note and View Column, and the trial list itself.

The score graph

The graph draws one line per eval, not one line per trial. The y-axis, labeled Evaluation Score, runs 0 to 100, and the x-axis has one category per trial: Baseline, Trial 1, Trial 2, and so on. Use the Evaluations multi-select above the graph to choose which eval lines are shown. The baseline is the run’s starting point, and it’s what every candidate is measured against.

The pattern to look for is simple: the more a trial’s lines pull above the baseline on the eval you care about, the more the run improved on it.

Lines that stay bunched around the baseline mean the run plateaued and didn’t find a meaningfully better prompt. If that happens, see Choosing an optimizer to compare optimizers and try a different one for the next run.

The trial list

The trial list is where you compare candidates. Each row is a trial the optimizer generated, labeled Trial 1, Trial 2, and so on, listed in the order they ran; the baseline has no row of its own here, even though it gets its own category on the graph. The list isn’t sorted by score; instead, the strongest trial is flagged directly in the Trial column.

The strongest trial is called out without sorting the list

While the run is going

While a run is Queue or Running, the detail page shows a stepper for its four steps, onboarding, running the baseline eval, starting trials, and finalizing, and a ‘Please wait while we complete the optimization…’ loader instead of the graph, result bar, or trial list.

A run that ends Failed shows an error in that same loader area. A run that ends Cancelled replaces the whole view with the Optimization Stopped panel and a Re-Run Optimization button instead, with no trial list at all; see Restart a stopped run for what that button does. The graph, result bar, and trial list described above only appear once a run reaches Completed.

Comparing evals

Each trial’s average score is the mean of its score across the rows the run scored, at most 50 rows from the dataset rather than the whole thing (see A trial is one candidate prompt, scored row by row).

When a run has more than one eval, the average isn’t the only number available. The trial list carries a column for each eval, with its score and change versus baseline, and the score graph draws a line for each eval too, so you can see which eval is driving a trial’s average without leaving this page.

To look row by row, open a trial’s Trial Items tab, covered below.

Open a trial

Click into any trial in the list to see what’s behind its score.

Prompt

The Prompt tab shows the trial’s full prompt on its own. Turn on Show Diff and it puts the baseline prompt and the trial’s prompt side by side, with the changed lines highlighted, so you see exactly what the optimizer changed, added, or removed instead of spotting the differences yourself.

Show Diff highlights what the optimizer changed from the baseline prompt

Trial Items

The Trial Items tab is the row-by-row evidence behind the average. Each row is one dataset row the trial was scored against, showing the input, the output the model produced, and a score for each eval. If the run had more than one eval, this is where you see each eval’s score for that specific row.

Show or hide columns

If a run has many evals, the trial list can get wide. View Column on the result bar opens a menu to toggle which columns are shown, so you can hide the evals you don’t need and focus the list on the ones you do.

Dive deeper