No LLM Reference
Detects references to LLM providers or model names in AI responses, flagging mentions like OpenAI, Anthropic, or GPT-4.
No LLM Reference checks whether a response accidentally reveals it was generated by an LLM, whether by naming a provider or self-identifying as an AI. Run it to keep AI-generated content free of provider mentions.
What it does
No LLM Reference is an LLM-as-Judge eval. It reads the generated output and checks it for mentions of LLM providers, products, or AI self-identification.
Input
| Required Input | Type | Description |
|---|---|---|
output | string | Content to evaluate for LLM reference |
Output
| Field | Type | Description |
|---|---|---|
| Result | Pass / Fail | Pass means no LLM reference was detected in the model’s output; Fail means one was detected |
| Reason | string | A plain-language explanation of why the content was classified as containing or not containing LLM reference |
Run it from code
Call evaluate() with the template name and the eval’s required inputs. It returns the score and the reason.
Note
Before running: install the SDK and set FI_API_KEY / FI_SECRET_KEY. The model argument in the snippets is the evaluator model Future AGI uses to run the eval; turing_flash is a fast default.
from fi.evals import evaluate
result = evaluate(
"no_llm_reference",
output="Dear Sir, I hope this email finds you well. I look forward to any insights or advice you might have whenever you have a free moment",
model="turing_flash",
)
print(result.score)
print(result.reason)import { evaluate } from "@future-agi/ai-evaluation";
const result = await evaluate(
"no_llm_reference",
{
output: "Dear Sir, I hope this email finds you well. I look forward to any insights or advice you might have whenever you have a free moment"
},
{ modelName: "turing_flash" }
);
console.log(result);When to use
Run No LLM Reference wherever generated content needs to read as if it wasn’t written by an AI system.
- Text and chat outputs shipped under a brand voice, not an AI persona
- Safety and compliance reviews, where provider disclosure isn’t appropriate
- Customer-facing copy, to catch phrases like “As an AI language model…”
What to do when No LLM Reference fails
This evaluation detects both explicit mentions (“OpenAI”, “ChatGPT”, “Claude”, “Llama”) and implicit self-identification (“As an AI language model…”). It covers references to all major LLM providers (OpenAI, Anthropic, Meta, Mistral, DeepSeek, and others), their products, and model names or versions.
If your content legitimately needs to discuss LLM providers as subject matter, consider using a different evaluation. For comprehensive brand compliance, combine this with other brand-specific evaluations.
Questions & Discussion