Is Helpful

Is Helpful checks whether a response actually solves the user’s problem, not just whether it’s well-formed. Run it wherever the response needs to move the user forward.

What it does

Is Helpful is an LLM-as-Judge eval. It reads the user query and the generated output, then checks whether the response is genuinely helpful.

Input

Output

Run it from code

Call evaluate() with the template name and the eval’s required inputs. It returns the score and the reason.

from fi.evals import evaluate

result = evaluate(
    "is_helpful",
    input="Why doesn't honey go bad?",
    output="Honey doesn't spoil because its low moisture and high acidity prevent the growth of bacteria and other microbes.",
    model="turing_flash",
)

print(result.score)
print(result.reason)

import { evaluate } from "@future-agi/ai-evaluation";

const result = await evaluate(
  "is_helpful",
  {
    input: "Why doesn't honey go bad?",
    output: "Honey doesn't spoil because its low moisture and high acidity prevent the growth of bacteria and other microbes."
  },
  { modelName: "turing_flash" }
);

console.log(result);

When to use

Run Is Helpful wherever a response needs to actually resolve what the user asked, not just sound plausible.

• Text and chat outputs answering a direct user question
• Customer support responses, to confirm the reply moves the user’s issue forward
• General quality checks, paired with more targeted evals for a fuller picture

What to do when Is Helpful fails

Ensure that both the input (user query) and output (AI response) parameters are provided; the helpfulness evaluation works best when the context of the request is clear. If evaluating complex responses, make sure the entire response is included.

Consider combining this with other evaluations like completeness for a more comprehensive assessment.