Environment FAQ & fixes
Common questions about environments, and fixes for the build and run problems you hit most
In this page
This page starts with fixes for the problems people hit most, grouped by when they happen, then answers common questions. If your answer isn’t here, reach out via support.
Fixes while building
| Symptom | Cause | Fix |
|---|---|---|
| Preflight, the check under Run preflight on the build form, reports Credentials need attention | An API key, an assistant or agent ID, or a GitHub App installation ID is wrong or can’t reach the agent | Correct the field it flags against the platform or GitHub, then run preflight again |
| The build reads Failed | The repository or folder can’t start the agent as shipped, for example there is no Docker Compose file or Dockerfile | Open the workspace and read the reason on the step that failed; add what is missing and build again. Supported agents lists the requirements |
| A chat agent from a repository fails to build | The agent has no way to accept a conversation | Expose an HTTP endpoint, in the agent’s own format or the OpenAI-compatible one, or a JSON WebSocket |
| The agent starts but can’t reach an API it depends on | Outbound traffic is blocked except for hosts you allow | Build again with the host under Additional egress domains |
| The agent starts but fails on a missing setting | A value it reads at startup wasn’t provided | Build again with the value under Environment values; values can’t be added to a built environment |
| The workspace warns Observability was not turned on | Turn on observability was ticked, but it couldn’t be set up for this environment; the warning gives the reason, such as the agent’s code could not be read | Nothing is lost: the environment built and runs as usual, without traces. If the reason says the run uses an existing agent, turn observability on from that agent’s page |
| A code upload is rejected for having too many files | The folder is larger than one upload can carry | Upload your agent’s own source, or a smaller subfolder |
Fixes while running
| Symptom | Cause | Fix |
|---|---|---|
| A run reads Failed | The run could not finish, for example the agent stopped answering or could not be reached | Open the run and read which calls errored and why; fix the cause, then tick the failed calls and use Re-run |
| A run stays in Queued or Provisioning | Other runs are ahead of it, or each scenario’s copy of the world is still being prepared | Leave it; it starts on its own. To give up on it, open the run and click Stop simulation |
| Fewer scenarios ran at once than Parallel worlds asked for | The agent can’t run several copies at once, for example it listens on a fixed network port or a value points at a fixed local address | Nothing is lost: the run says why, and the remaining scenarios wait for a free world |
| Calls show as Error | Something failed around the agent, such as a connection that never came up, so there is no verdict | Calls with an error never count against the agent; open one to see what failed, then tick those calls and use Re-run |
| A finished run has no scores | No evaluations were on the environment when it ran | Add evaluations, then use Run evals on the run to grade its finished calls |
| The chat orchestration box is greyed out | The environment is still being built | Wait for Ready; the chat opens then |
| Chat orchestration refuses a request | The request changes the agent or how it is reached: its phone number, IDs, credentials, prompt or code | Make that change where the agent lives; see Chat orchestration |
Building
Which agents can I connect?
An agent on GitHub, a folder of agent code from your machine, a voice agent on Vapi or Retell AI, a chat agent on Retell AI, or any voice agent that answers a phone number. Supported agents lists what each needs.
Does building an environment change my agent?
No. The builder reads your agent and runs it as it is. It never edits its code, its prompt or its platform configuration, and corrections you make through chat orchestration change only the environment’s contract.
Can I change the source or the environment values after building?
No. Everything on the Settings tab is fixed at build time except the name. Build a new environment from the new source or values.
How do I know a build is still moving?
The workspace opens as soon as the build starts and fills in stage by stage, with each step marked as it finishes. A build that stops reads Failed with the reason on its step; one that is still working keeps advancing through the steps.
Scenarios and contracts
The builder read my agent wrongly. How do I fix it?
Tell chat orchestration what is wrong and what is true, then check that the tool or rule shows on Overview. Review the contract lists what to check.
Can I add more scenarios after the build?
Yes. Ask through chat orchestration, for example “add five scenarios where the caller is in a hurry”. Each one is checked the same way as the scenarios written at build time before it is kept.
Running
Do I need evaluations to run?
No, but without them a run records conversations and nothing grades them beyond each scenario’s sub-goals. Add evaluations covers choosing them.
Why run a scenario more than once?
A conversation with an AI agent is not deterministic, so one pass can succeed by chance. Set trials or repeats when you run a simulation; Reliability across trials on a run’s Analytics tab then gives the pass rate with its 95% range.
Do scenario edits change runs I already finished?
No. The next run plays the edited scenarios, and finished runs keep the ones they played.
Questions & Discussion