Guardrail fires on the wrong requests

A guardrail check can go wrong in two directions: it blocks or warns on requests that were fine, or it lets through content it should have caught. Either way, the fix is the same set of levers, worked in order on the one check that’s causing it.

Go to Gateway > Guardrails. Overview, Rules, Analytics, Feedback, Test, and Logs are all tabs on that section; Analytics is where you diagnose which check is misfiring, and Rules is where you open that check’s card to edit it.

Find the check that’s firing

Checks show up under Rule in the Top Triggered Rules table alongside Triggers, Block, and Warn counts. Open Gateway > Guardrails > Analytics to see it:

• Over-blocking: a high Block count points to the check to look at first, not proof on its own that it’s firing on requests that were fine
• Letting through: if a specific kind of content, like PII or a prompt injection, is getting through, open the check meant to catch it

Switch Action to Warn or Log while you tune

Switch Action from Block to Warn or Log before you touch the Confidence Threshold below, so a check you’re still tuning doesn’t stop real traffic. Go to Gateway > Guardrails > Rules, find the check’s card, and click its pencil icon button to open the dialog (see Turn on a guardrail for the fuller walkthrough of setting these up). In the dialog’s Action select:

• Block stops the request, and counts toward that check’s Block total in Top Triggered Rules
• Warn lets the request through, and counts toward that check’s Warn total in Top Triggered Rules
• Log lets the request through too, and still counts toward Triggers, but the check’s Block and Warn totals stay flat, for a quieter pass while you compare several changes

Mask is also on the select, but it’s a different use case outside this tuning flow. Set Action back to Block once you’re satisfied with where the threshold lands.

Move its Confidence Threshold

In the same dialog, find Confidence Threshold, a slider marked at 0.0, 0.5, and 1.0. Its untouched value is 0.8. Future AGI Eval and Presidio PII checks don’t render this slider at all, so this lever isn’t available for them. For what action and threshold actually do to a request, see Understanding Protect.

Raise the threshold and the check catches less, so move it up if the check is blocking or warning on requests that were fine. Lower it and the check catches more, so move it down if it’s letting through content it should have caught. As a first move, try adjusting it by about 0.05 to 0.1, then re-test before adjusting further.

Click Save in the dialog to stage the change, then back on the Rules tab, click Save & Activate on the “You have unsaved changes. Save to push guardrail config to the gateway.” banner; the change doesn’t reach the gateway until you do. If you re-test and nothing’s changed, see Guardrail changes not taking effect.

Re-test after each change

After each change, switch to Gateway > Guardrails > Test (see Test a guardrail for what it shows) and click the example chip that matches the case you’re chasing, whether that’s PII, an injection attempt, secrets, or toxic content, then run it. The result chip in the Result card reads BLOCKED (446), WARNING (246), or OK with the status code; that’s the verdict to check against what you meant to happen. Re-running that chip after every threshold or Action change is how you confirm the change did what you meant, instead of stacking up several changes and losing track of which one mattered.

Record the verdict as feedback

From Gateway > Guardrails > Logs, open the request the check got wrong in its detail drawer. The drawer’s Guardrails tab only appears when at least one check fired on that request; a request that nothing caught has no feedback controls at all, so it can’t be marked False Negative from its own drawer. On the drawer’s Guardrails tab, one set of feedback controls appears for each check that fired on that request; find the set for the check you’re tuning and mark it False Positive if it fired on a request that was fine, or False Negative if it let through something it should have caught, then press Submit Feedback. That verdict rolls up into Gateway > Guardrails > Feedback alongside every other correction submitted for that check.

If it still fires wrong

If the check still fires wrong after switching Action, moving the Confidence Threshold, and re-testing, the fix may be more than this check can offer on its own. Check whether a different check in the Top Triggered Rules table above is the actual source, or contact support@futureagi.com with the check’s name, the Action and threshold you tried, and an example request it still gets wrong.

Dive deeper