Replay
Reproduce a real production bug as a test, instead of guessing at a synthetic one
What replay is
Replay takes a real production conversation from Observe and rebuilds it as two things you can rerun: a scenario made from the transcript, and an agent definition carrying the configuration that call ran on. Say a caller got quoted the wrong refund window last week: you pick that exact conversation, rebuild it, and rerun it to confirm your fix holds. Instead of guessing at a synthetic case, you reproduce the real one.
Note
Replay reads what Observe already recorded, so your app has to be sending traces before any of this is available. Replaying a whole conversation additionally needs those traces to carry a session ID, and voice replay needs voice observability on the original call.
Session replay and trace replay
You choose how much of the production data becomes one conversation.
Session
A whole session, every trace under one session_id in order, replays as a single multi-turn conversation. Reach for it when you want to rerun full production conversations end to end.
Trace
Each selected trace replays as its own one-turn conversation, an input and an output. Reach for it when you want to replay individual calls or single-turn interactions.
Chat and voice
Replay works in both channels, and voice carries more of the original setup across.
Chat replay
Rebuilds the conversation from the production transcripts and runs it against your agent.
Voice replay
Goes further: it pulls the original voice setup (system prompt, assistant settings, and provider config) from the production call, so the replayed call runs on the same configuration as the original.
Vapi is the one config extraction is built around. Retell and Bland.ai calls replay too, though what you get back to compare afterwards is the transcript rather than the full call. If your calls run on any other stack, voice replay can’t reconstruct the original configuration, so replay the conversation as chat instead.
What a replay produces
The recreated agent definition reproduces the agent as it behaved in production, which makes it your baseline, not your fix. You edit it, or point the run at a newer version, and the difference between the two runs is what your change did.
Once the run finishes, you compare the replayed conversation with the original side by side, transcripts, metrics, and for voice the audio, so you can see exactly what your change moved.
A replayed failure becomes a regression test
A production failure that only happened once can slip away. A replayed scenario is an ordinary scenario, so keep it alongside the others you run on every change and the exact conversation that broke becomes something every future version has to pass.
Keep exploring
Questions & Discussion