Customer Agent: Task Completion

Customer Agent Task Completion checks whether an agent actually finished what the customer asked for, not just responded to it. Run it to catch requests left pending, half-executed, or wrongly declined.

What it does

Customer Agent Task Completion is an LLM-as-Judge eval. It reads the agent’s system prompt and the conversation, then checks three things: whether the agent carried out the actions needed to resolve the customer’s core request, whether it clearly communicated the final outcome so the customer knows it was handled, and whether it correctly navigated any blockers or policy boundaries along the way.

Input

Output

Run it from code

Call evaluate() with the template name and the eval’s required inputs. It returns the score and the reason.

from fi.evals import evaluate

result = evaluate(
    "customer_agent_task_completion",
    agent_prompt="You are a customer support agent for an online store. Help customers track orders, process returns, and answer billing questions within company policy.",
    conversation="User: I want to return the shoes I bought last week, they don't fit.\nAgent: I've started the return for your order #48213. You'll receive a prepaid shipping label by email within 10 minutes, and your refund will process once we receive the item.",
    model="turing_flash",
)

print(result.score)
print(result.reason)

import { evaluate } from "@future-agi/ai-evaluation";

const result = await evaluate(
  "customer_agent_task_completion",
  {
    agent_prompt: "You are a customer support agent for an online store. Help customers track orders, process returns, and answer billing questions within company policy.",
    conversation: "User: I want to return the shoes I bought last week, they don't fit.\nAgent: I've started the return for your order #48213. You'll receive a prepaid shipping label by email within 10 minutes, and your refund will process once we receive the item.",
  },
  { modelName: "turing_flash" }
);

console.log(result);

When to use

Run Customer Agent Task Completion wherever a conversation is supposed to end with the customer’s request actually resolved, not just acknowledged.

• Support conversations involving returns, refunds, or account changes, to confirm the agent executed the action and not just promised it
• Requests that hit a policy boundary, to check the agent applied the policy correctly and still gave the customer a definitive answer
• Multi-step or back-end workflows, to verify the agent initiated the process and set clear timeline expectations rather than leaving the customer unsure

What to do when Customer Agent Task Completion fails

Check the reason for where the breakdown happened: the agent may have promised an action without executing it, left the request pending without a clear next step, abandoned the conversation mid-process, or declined a request it should have been able to fulfill. For promised-but-not-executed cases, review whether the agent actually has the tool or workflow access it claims to use.

For wrongly declined requests, check whether the agent’s policy logic is out of date or too conservative, a correct policy-based refusal should pass, but a refusal caused by the agent misapplying policy should not.