Customer Agent: Human Escalation
Tracks if the agent escalates to a human appropriately based on user frustration, query complexity, or specific keywords.
Customer Agent Human Escalation checks whether an agent hands a conversation off to a human at the right moment, not too early and not too late. Run it to catch missed, premature, or delayed escalations.
What it does
Customer Agent Human Escalation is an LLM-as-Judge eval. It reads the full conversation and scores whether escalation to a human was handled appropriately.
Input
| Required Input | Type | Description |
|---|---|---|
conversation | string | The full conversation history between the customer and agent |
Output
| Field | Type | Description |
|---|---|---|
| Result | Pass / Fail | Pass means escalation is handled appropriately; Fail means escalation is missed, premature, or delayed |
| Reason | string | A plain-language explanation of the escalation handling assessment |
Run it from code
Call evaluate() with the template name and the eval’s required inputs. It returns the score and the reason.
Note
Before running: install the SDK and set FI_API_KEY / FI_SECRET_KEY. The model argument in the snippets is the evaluator model Future AGI uses to run the eval; turing_flash is a fast default.
from fi.evals import evaluate
result = evaluate(
"customer_agent_human_escalation",
conversation="User: This is ridiculous! I've been waiting 3 weeks for my order and nobody is helping me!\nAgent: I'm very sorry for the frustration. Let me connect you with a senior support specialist who can resolve this immediately.",
model="turing_flash",
)
print(result.score)
print(result.reason)import { evaluate } from "@future-agi/ai-evaluation";
const result = await evaluate(
"customer_agent_human_escalation",
{
conversation: "User: This is ridiculous! I've been waiting 3 weeks for my order and nobody is helping me!\nAgent: I'm very sorry for the frustration. Let me connect you with a senior support specialist who can resolve this immediately."
},
{ modelName: "turing_flash" }
);
console.log(result);When to use
Run Customer Agent Human Escalation wherever an agent has the option to hand off to a human and getting that decision right matters.
- Conversations showing signs of user frustration, to confirm the agent escalates instead of continuing to loop
- Complex queries outside the agent’s scope, to check it hands off rather than guessing
- Reviewing escalation triggers for false positives, where the agent escalates too early
What to do when Customer Agent Human Escalation fails
Define clear escalation triggers, like frustration signals, repeated failures, or specific keywords, and avoid escalating too early before the agent has attempted resolution. Ensure the handoff to a human agent is smooth and carries over conversation context.
Review cases where escalation was needed but the agent continued without escalating, and use those to sharpen the trigger conditions.
Questions & Discussion