Runs & results

What a run is

A run test bundles everything one simulation needs: an agent version, the scenarios to play, and the evals that score the result. The personas come along inside the scenarios, attached when each one was built, so you don’t pick them again here. Run one version of your support agent against a refund scenario carrying a frustrated persona, scored by a resolution eval, and you have one run test.

You create and start one from the dashboard: Run a voice simulation and Run a chat simulation walk through it end to end.

From run to calls

Starting a run test creates an execution, one attempt at playing the whole bundle, and the execution fans out into calls: one call per scenario, or one per row of the scenario’s table. Each call runs its conversation end to end.

%%{init: {"flowchart": {"curve": "basis", "rankSpacing": 70, "nodeSpacing": 45, "padding": 18}}}%%
flowchart TB
  RT["Run test<br/>one agent version, the scenarios, the evals"] --> EX(["Execution"])
  EX --> C1["Call<br/>refund, frustrated caller"]
  EX --> C2["Call<br/>refund, polite regular"]
  EX --> C3["Call<br/>booking change"]
  C1 --> REC["Each call leaves a transcript,<br/>metrics, and eval results"]
  C2 --> REC
  C3 --> REC

An execution reports where it is. It moves through pending, running, and evaluating, then finishes as completed, or as failed or cancelled when it stops early; cancelling is the brief state while a stop you requested takes effect.

The results each call carries

Every call is the record of one conversation. It holds:

• The transcript, turn by turn, with the speaker role on each turn
• The recording, for voice calls
• Conversation metrics, like latency, talk ratio, and cost
• Eval results, a score per metric for that call, and tool-call results when the run test was created with tool evaluation switched on

The exact metric fields and speaker roles live in the Call metrics reference.

Reruns and snapshots

From a run’s results view, covered in Explore results, you can rerun a whole call or only its evals, for example after changing an eval config. A rerun doesn’t overwrite the earlier result: the previous run is snapshotted, so you can compare a call before and after a change instead of losing the baseline.

Runs are comparable

Because a run pins an agent version and replays the same scenarios, two runs differ only by what you changed. That makes their scores directly comparable, so you can trend quality across versions and catch a regression before it ships.

Keep exploring