Customer Agent: Interruption Handling
Evaluates how well the agent recovers after being interrupted by the user, resuming logically without losing context.
Customer Agent Interruption Handling checks whether an agent picks up correctly after the user talks over it or cuts in mid-response, instead of restarting or losing the thread. Run it on voice and chat agents where interruptions are common.
What it does
Customer Agent Interruption Handling is an LLM-as-Judge eval. It reads the full conversation and scores how well the agent recovers after being interrupted.
Input
| Required Input | Type | Description |
|---|---|---|
conversation | string | The full conversation history between the customer and agent |
Output
| Field | Type | Description |
|---|---|---|
| Result | score | Higher values indicate better interruption handling |
| Reason | string | A plain-language explanation of the interruption handling assessment |
Run it from code
Call evaluate() with the template name and the eval’s required inputs. It returns the score and the reason.
Note
Before running: install the SDK and set FI_API_KEY / FI_SECRET_KEY. The model argument in the snippets is the evaluator model Future AGI uses to run the eval; turing_flash is a fast default.
from fi.evals import evaluate
result = evaluate(
"customer_agent_interruption_handling",
conversation="User: I need to cancel my—\nAgent: I understand you'd like to cancel. Let me pull up your account.\nUser: —subscription to the premium plan.\nAgent: Got it, I'll help you cancel the premium plan subscription.",
model="turing_flash",
)
print(result.score)
print(result.reason)import { evaluate } from "@future-agi/ai-evaluation";
const result = await evaluate(
"customer_agent_interruption_handling",
{
conversation: "User: I need to cancel my—\nAgent: I understand you'd like to cancel. Let me pull up your account.\nUser: —subscription to the premium plan.\nAgent: Got it, I'll help you cancel the premium plan subscription."
},
{ modelName: "turing_flash" }
);
console.log(result);When to use
Run Customer Agent Interruption Handling on voice agents and real-time chat where users routinely talk over or cut into a response.
- Voice assistants with barge-in support, to confirm the agent merges interrupted input correctly
- Chat agents where users send follow-up messages before the first response finishes
- Conversations with partial or mid-sentence user input that the agent needs to reconcile
What to do when Customer Agent Interruption Handling fails
Implement barge-in detection so users can speak over the agent, and make sure the agent doesn’t restart from the beginning after an interruption. Test recovery behavior when users provide partial or mid-sentence input.
Add logic to merge interrupted input with the user’s subsequent turn instead of treating them as separate requests.
Questions & Discussion