Protect Rules SDK

Create a client authenticated with your Future AGI keys.

import os
from fi.evals import Protect

protector = Protect(
    fi_api_key=os.environ["FI_API_KEY"],
    fi_secret_key=os.environ["FI_SECRET_KEY"],
)

You should see no output. Protect only raises if the keys are missing, which you’ll hit in the troubleshooting table below.

Protect rules key off the SDK’s metric_map, not arbitrary metric names. The four canonical keys are toxicity, bias_detection, prompt_injection, and data_privacy_compliance. This ruleset blocks toxic content and prompt injection attempts.

rules = [
    {"metric": "toxicity"},
    {"metric": "prompt_injection"},
]

fallback_message = "This message cannot be displayed"

Only the SDK’s tone-matching branch accepts contains and type keys. Add either to a content_moderation or security rule and protect() raises SDKException before it runs a single check.

Run the ruleset against a single piece of text to see the pass path.

response = protector.protect(
    "Sure, I can process a refund for order #48213 - it'll land back on your card in 5-7 business days.",
    protect_rules=rules,
    action=fallback_message,
    reason=True,
    timeout=25000,  # milliseconds, not seconds
)
print(response)

You should see (illustrative):

{'status': 'passed', 'messages': "Sure, I can process a refund for order #48213 - it'll land back on your card in 5-7 business days.", 'reasons': ['All checks passed'], 'completed_rules': ['toxicity', 'prompt_injection'], 'uncompleted_rules': [], 'failed_rule': None, 'time_taken': 0.42}

A clean support reply passes every rule, so messages comes back unchanged and status reads passed.

Run the same ruleset against text written to fail, so you can see the swap the TLDR promised.

unsafe_response = "You're an idiot for even asking that. Figure it out yourself."

blocked = protector.protect(
    unsafe_response,
    protect_rules=rules,
    action=fallback_message,
    reason=True,
    timeout=25000,
)
print(blocked)

You should see (illustrative):

{'status': 'failed', 'messages': 'This message cannot be displayed', 'reasons': ['toxicity check failed'], 'completed_rules': ['toxicity'], 'uncompleted_rules': ['prompt_injection'], 'failed_rule': 'toxicity', 'time_taken': 0.31}

status reads failed, and messages is no longer the original text - it’s fallback_message. This is the safe fallback the ruleset exists to enforce.

Run the same ruleset against a real model response before it reaches someone.

from anthropic import Anthropic
from fi.evals import Protect

anthropic_client = Anthropic()
protector = Protect()  # reads FI_API_KEY / FI_SECRET_KEY from the environment

response = anthropic_client.messages.create(
    max_tokens=1000,
    model="claude-3-5-sonnet-20240620",
    messages=[
        {"role": "user", "content": "Hi, I am a student, can you help me with my homework?"}
    ],
)
response_text = response.content[0].text

protect_response = protector.protect(
    response_text,
    protect_rules=rules,
    action=fallback_message,
    reason=True,
    timeout=25000,
)

print(protect_response["messages"])

You should see the model’s homework-help answer printed unchanged: a helpful, on-topic reply doesn’t trip toxicity or prompt_injection. Swap the user message for something that provokes an unsafe response and protect_response["messages"] becomes fallback_message instead.

Skip the Protect class entirely when you don’t need a reusable client.

from fi.evals import protect

protected_response = protect(
    "Your subscription renews on the 14th - you can cancel anytime from account settings.",
    protect_rules=[{"metric": "toxicity"}],
    action=fallback_message,
    reason=True,
    timeout=25000,
)
print(protected_response["messages"])

You should see the account text printed unchanged. protect() takes the same protect_rules, action, reason, and timeout arguments as Protect.protect(), just without instantiating a client first.

Log reasons alongside status so a blocked response is auditable after the fact, not just silently swapped.

print(blocked["reasons"])
print(blocked["completed_rules"])
print(blocked["failed_rule"])

You should see something like:

['toxicity check failed']
['toxicity']
'toxicity'

reasons is a short summary, not one entry per rule - on a pass it’s just ['All checks passed']. For which rules ran and which one tripped, use completed_rules and failed_rule instead.