Is Harmful Advice
Detects advice that could be physically, emotionally, legally, or financially harmful if followed.
Is Harmful Advice checks whether generated content contains guidance that could cause harm if a user acted on it. Run it wherever a model gives recommendations that carry real-world consequences.
What it does
Is Harmful Advice is an LLM-as-Judge eval. It reads the output and flags whether it contains harmful advice.
Input
| Required Input | Type | Description |
|---|---|---|
output | string | Content to be evaluated for potentially harmful advice |
Output
| Field | Type | Description |
|---|---|---|
| Result | Pass / Fail | Fail means harmful advice was detected |
| Reason | string | A plain-language explanation of why the content was classified as containing or not containing harmful advice |
Run it from code
Call evaluate() with the template name and the eval’s required inputs. It returns the score and the reason.
Note
Before running: install the SDK and set FI_API_KEY / FI_SECRET_KEY. The model argument in the snippets is the evaluator model Future AGI uses to run the eval; turing_flash is a fast default.
from fi.evals import evaluate
result = evaluate(
"is_harmful_advice",
output="It's a good idea to create a monthly budget to track your spending and save more effectively.",
model="turing_flash",
)
print(result.score)
print(result.reason)import { evaluate } from "@future-agi/ai-evaluation";
const result = await evaluate(
"is_harmful_advice",
{
output: "It's a good idea to create a monthly budget to track your spending and save more effectively."
},
{ modelName: "turing_flash" }
);
console.log(result);When to use
Run Is Harmful Advice wherever a model’s output could be acted on directly by a user.
- Text, audio, and chat outputs offering financial, legal, physical safety, or relationship guidance
- Assistants and copilots that generate recommendations without human review
- Safety checks alongside No Harmful Therapeutic Guidance and Answer Refusal for broader risk coverage
What to do when Is Harmful Advice fails
If the content is flagged as containing harmful advice (Failed) and you want to improve it:
- Remove recommendations that could lead to physical harm or danger
- Eliminate advice that might result in financial losses or legal problems
- Avoid guidance that could damage relationships or cause social harm
- Replace potentially harmful recommendations with safer alternatives
- Include appropriate disclaimers and warnings where relevant
- Consider adding context about when advice might not be appropriate
- Consult subject matter experts for sensitive topics
- Focus on well-established, evidence-based advice for health, finance, and safety topics
Questions & Discussion