Understanding Environments

What an environment is

An environment wraps one of your agents in everything a test needs: a world to act in, scenarios to get through, and a way to grade each call. Your agent is tested as it is, with its own instructions and tools, while the data it works on is a test version made for the purpose, so a test can change records, fail, and start over without touching anything your customers rely on. Because every call is checked against what the agent did, not just what it said, you get a repeatable read on whether a change made your agent better or worse.

How an environment works

Four things go into an environment:

• the contract, the record of what your agent is: what it’s for, the tools it calls, the rules it has to keep, and the use cases it handles
• the world, a test version of the database your agent reads and changes, rebuilt from its code and seeded with the records the scenarios need, like a rider with a saved card or a booking that has already been cancelled. It’s never your live data
• the tools, your agent’s own actions (look up a ride, send a code, take a payment), connected to the world so a tool call in a test really changes a record
• the scenarios, the test cases, each starting from the same clean copy of the world

You don’t assemble any of it. The builder, the Future AGI agent that reads your agent, makes all four from what you connect, and a simulated caller plays each scenario against your agent: it takes on the scenario’s persona and works through its situation, turn by turn.

In a call, your agent answers the simulated caller and calls its tools, and those tools work on the world. When the call ends, the scenario’s sub-goals decide whether the agent did its job. Each sub-goal is one thing the agent is checked on, like whether the booking was cancelled, and is settled from the world or the conversation. The next call starts from a clean copy again, so its result never depends on the one before. Build an environment covers connecting each kind of agent.

The test-and-fix loop

Say your agent books rides by phone. You connect its code repository and the builder makes an environment for it. The loop looks like this:

1. The scenarios include a caller who wants to change their pickup address
2. You run it, and the call fails: the agent says the pickup is updated, but the booking in the world still has the old address
3. You change the agent so it saves the new address before telling the caller it’s done
4. You run the same scenario again, and it passes

A transcript alone would have read as a pass the first time; only the check against the world caught it. Same test, changed agent, so when the result flips it’s proof the fix worked, not a hunch.

When your agent runs on a platform

An agent on Vapi, Retell AI, LiveKit, or one that answers a phone number is tested exactly where it runs: the environment reaches it on its platform, the same way your customers do, and reads how it’s set up there. Its data stays on that platform, so the environment holds the contract, the connection to your agent, and the scenarios, and every call is judged on the conversation itself: what the agent said and what it agreed to do.

Where environments fit

Evaluation supplies the evaluations you can add to an environment, which grade every call alongside the scenarios’ own sub-goals.

And two capabilities inside environments shape what gets tested:

• Contracts record what the builder understood about your agent, and every scenario and grade follows from them
• Chat orchestration lets you ask the builder about what it made and steer it in plain words

Keep exploring