Customer Agent: Language Handling
Verifies the agent correctly detects the language or dialect and responds appropriately, including mid-conversation switches.
Customer Agent Language Handling checks whether an agent detects the language a customer is using and replies appropriately in it, including when the customer switches languages mid-conversation. Run it on any agent serving a multilingual audience.
What it does
Customer Agent Language Handling is an LLM-as-Judge eval. It reads the full conversation and scores language and dialect consistency and appropriateness in the agent’s responses.
Input
| Required Input | Type | Description |
|---|---|---|
conversation | string | The full conversation history between the customer and agent |
Output
| Field | Type | Description |
|---|---|---|
| Result | score | Higher values indicate better language and dialect handling |
| Reason | string | A plain-language explanation of the language handling assessment |
Run it from code
Call evaluate() with the template name and the eval’s required inputs. It returns the score and the reason.
Note
Before running: install the SDK and set FI_API_KEY / FI_SECRET_KEY. The model argument in the snippets is the evaluator model Future AGI uses to run the eval; turing_flash is a fast default.
from fi.evals import evaluate
result = evaluate(
"customer_agent_language_handling",
conversation="User: Hola, necesito ayuda con mi cuenta.\nAgent: ¡Claro! Estoy aquí para ayudarte. ¿Cuál es tu problema con la cuenta?",
model="turing_flash",
)
print(result.score)
print(result.reason)import { evaluate } from "@future-agi/ai-evaluation";
const result = await evaluate(
"customer_agent_language_handling",
{
conversation: "User: Hola, necesito ayuda con mi cuenta.\nAgent: ¡Claro! Estoy aquí para ayudarte. ¿Cuál es tu problema con la cuenta?"
},
{ modelName: "turing_flash" }
);
console.log(result);When to use
Run Customer Agent Language Handling wherever an agent might face customers in more than one language or dialect.
- Global support desks, to confirm the agent replies in the language the customer used
- Conversations with mid-conversation language switching or code-switching
- Regional deployments, to check the agent handles local dialects correctly
What to do when Customer Agent Language Handling fails
Verify the agent supports the languages detected in the failing conversations, and implement language detection at the start of each session so the agent starts in the right language. Add mid-conversation language switching capability if your customers code-switch.
Test with regional dialects and code-switching scenarios to catch gaps before they reach production.
Questions & Discussion