Customer Agent: Interruption Handling

Evaluates how well the agent recovers after being interrupted by the user, resuming logically without restarting or losing context. 

result = evaluator.evaluate(
    eval_templates="customer_agent_interruption_handling",
    inputs={
        "conversation": "User: I need to cancel my—\nAgent: I understand you'd like to cancel. Let me pull up your account.\nUser: —subscription to the premium plan.\nAgent: Got it, I'll help you cancel the premium plan subscription."
    },
    model_name="turing_flash"
)

print(result.eval_results[0].output)
print(result.eval_results[0].reason)
import { Evaluator, Templates } from "@future-agi/ai-evaluation";

const evaluator = new Evaluator();

const result = await evaluator.evaluate(
  "customer_agent_interruption_handling",
  {
    conversation: "User: I need to cancel my—\nAgent: I understand you'd like to cancel. Let me pull up your account.\nUser: —subscription to the premium plan.\nAgent: Got it, I'll help you cancel the premium plan subscription."
  },
  {
    modelName: "turing_flash",
  }
);

console.log(result);
Input
Required InputTypeDescription
conversationstringThe full conversation history between the customer and agent
Output
FieldDescription
ResultReturns a numeric score from 0 to 100, where higher values indicate better interruption handling
ReasonProvides a detailed explanation of the interruption handling assessment

What to Do When Interruption Handling Score is Low

  • Implement barge-in detection to allow users to speak over the agent
  • Ensure the agent does not restart from the beginning after an interruption
  • Test recovery behavior when users provide partial or mid-sentence input
  • Add logic to merge interrupted input with subsequent user turns

Comparing Interruption Handling with Similar Evals

  • Customer Agent: Termination Handling: Interruption Handling evaluates smooth recovery from user interruptions, while Termination Handling tracks abrupt or unexpected conversation endings.
  • Customer Agent: Context Retention: Interruption Handling focuses on resuming correctly after being cut off, while Context Retention evaluates memory of earlier conversation details.
Was this page helpful?

Questions & Discussion