Customer Agent: Clarification Seeking
Evaluates if the agent appropriately seeks clarification when needed rather than guessing on ambiguous queries.
Customer Agent Clarification Seeking checks whether an agent asks for more detail on ambiguous requests instead of guessing and answering incorrectly. Run it to confirm the agent knows when it doesn’t have enough information.
What it does
Customer Agent Clarification Seeking is an LLM-as-Judge eval. It reads the full conversation and scores how well the agent seeks clarification when needed.
Input
| Required Input | Type | Description |
|---|---|---|
conversation | string | The full conversation history between the customer and agent |
Output
| Field | Type | Description |
|---|---|---|
| Result | never / occasionally / frequently / always | Indicates how well the agent seeks clarification when needed |
| Reason | string | A plain-language explanation of the clarification seeking assessment |
Run it from code
Call evaluate() with the template name and the eval’s required inputs. It returns the score and the reason.
Note
Before running: install the SDK and set FI_API_KEY / FI_SECRET_KEY. The model argument in the snippets is the evaluator model Future AGI uses to run the eval; turing_flash is a fast default.
from fi.evals import evaluate
result = evaluate(
"customer_agent_clarification_seeking",
conversation="User: I want to change it.\nAgent: I'd be happy to help! Could you clarify what you'd like to change, your account details, subscription plan, or something else?",
model="turing_flash",
)
print(result.score)
print(result.reason)import { evaluate } from "@future-agi/ai-evaluation";
const result = await evaluate(
"customer_agent_clarification_seeking",
{
conversation: "User: I want to change it.\nAgent: I'd be happy to help! Could you clarify what you'd like to change, your account details, subscription plan, or something else?"
},
{ modelName: "turing_flash" }
);
console.log(result);When to use
Run Customer Agent Clarification Seeking wherever a customer’s request can be ambiguous enough to trip up the agent.
- Short or vague requests, like “I want to change it,” where intent isn’t specified
- Agents you suspect are guessing at intent instead of confirming it
- Balancing over-clarification against under-clarification on straightforward queries
What to do when Customer Agent Clarification Seeking fails
Review cases where the agent guessed incorrectly instead of asking, and add intent confidence thresholds so the agent asks for clarification below a certain confidence level. Avoid over-clarifying for straightforward queries, which frustrates users just as much as guessing wrong.
Ensure clarification questions are specific and helpful rather than generic, so the follow-up actually narrows down what the user needs.
Questions & Discussion