Replay chat sessions

Replay turns one production conversation into a scenario you can run in Simulation. This guide is the how-to half: finding the session, what replay carries over from it, and what you get back once you run it. For what replay is, and how session replay differs from trace replay, read the concept page first.

Find the session

Open Explore dashboard and look for the conversation you actually want to reproduce: a chat that gave a wrong answer, lost the thread partway through, or escalated when it shouldn’t have. Observe groups a conversation’s turns under one session.id, and that grouping is exactly what replay reads, so a conversation split across several session IDs won’t come back as a single scenario.

Start the replay

From the session, start a replay. Future AGI reads its turns in order and turns them into a scenario: one row that plays out the same conversation, in the order it actually happened. It also creates an agent definition to hold that scenario, since every scenario has to belong to one.

The agent definition it creates is just a place for the scenario to live; you still point the run at whichever version of your own agent you want to test.

What carries over

• The conversation. Every turn of the session, in order, becomes the scenario’s script. Run it and the simulator works through the same exchange your production user had, not one it invents
• No voice-only detail. Provider configuration, recordings, and call-level metrics belong to voice; a chat replay carries none of it, since chat calls never had it to begin with

Run it

A replayed scenario is an ordinary scenario, so run it the same way as any chat run: create a simulation against it, then drive it from your own code with the SDK.

pip install agent-simulate

import asyncio
from fi.simulate import TestRunner, AgentInput

async def customer_support_agent(input: AgentInput) -> str:
    user_message = input.new_message["content"] if input.new_message else ""
    return await my_agent.respond(user_message)

async def main():
    runner = TestRunner()
    await runner.run_test(
        run_test_name="Replaying chat_1001",  # your run's name, exactly
        agent_callback=customer_support_agent,
    )

asyncio.run(main())

Run a chat simulation covers the callback in full, including returning an AgentResponse when your agent’s tool calls need to show up in the results.

What comes back

The run produces an execution with the replayed conversation’s transcript and metrics, same as any chat run, readable from Calls & transcripts the same way you’d read any other one.

Once it’s finished, open the call and use Compare with baseline chat. It lines the replayed conversation up against the original side by side, transcripts and eval scores together, so you can see exactly what your change moved instead of just that a number went up.

Dive deeper