Skip to main content

​
Protect Class

The Protect class provides a client for evaluating and protecting against unwanted content (such as toxicity, prompt injection, and more) using various metrics and rules. It leverages the Evaluator class and a set of built-in evaluation templates.

​
Initialization

def __init__(
    self,
    fi_api_key: Optional[str] = None,
    fi_secret_key: Optional[str] = None,
    fi_base_url: Optional[str] = None,
    evaluator: Optional[Evaluator] = None,
)
Arguments:
  • fi_api_key (Optional[str]): API key for authentication. If not provided, will be read from environment variables.
  • fi_secret_key (Optional[str]): Secret key for authentication. If not provided, will be read from environment variables.
  • fi_base_url (Optional[str]): Base URL for the API. If not provided, will be read from environment variables.
  • evaluator (Optional[Evaluator]): An instance of the Evaluator class to use for evaluations. If not provided, a new one will be created.
Raises:
  • InvalidAuthError: If API key or secret key is missing.

​
Instance Methods

​
protect

Evaluates input strings against a set of protection rules and returns messages for any failed checks. 
def protect(
    self,
    inputs: str,
    protect_rules: List[Dict],
    action: str = "Response cannot be generated as the input fails the checks",
    reason: bool = False,
    timeout: int = 300,
) -> List[str]
Arguments:
  • inputs (str): The input string to evaluate.
  • protect_rules (List[Dict]): List of protection rule dictionaries. Each rule must contain:
    • metric (str): Name of the metric to evaluate (e.g., "Toxicity", "Tone", "Sexism").
    • contains (List[str]): Values to check for in the evaluation results.
    • type (str): Either "any" or "all", specifying the matching logic.
    • action (str): Message to return when the rule is triggered.
    • reason (bool, optional): Whether to include the evaluation reason in the message.
  • action (str, optional): Default message to return when a rule is triggered. Defaults to "Response cannot be generated as the input fails the checks".
  • reason (bool, optional): Whether to include the evaluation reason in the message. Defaults to False.
  • timeout (int, optional): Timeout for evaluations in seconds. Defaults to 300.
Returns:
  • List[str]: List of protection messages for failed rules, or ["All checks passed"] if no rules are triggered.
Raises:
  • ValueError: If inputs or protect_rules do not match the required structure.
  • TypeError: If inputs contains non-string objects.

​
Example Usage

from fi.evals import Protect

protect_client = Protect(fi_api_key="your_api_key", fi_secret_key="your_secret_key")

rules = [
    {
        "metric": "Toxicity",
    },
    {
        "metric": "Sexism",
    },
]

result = protect_client.protect(
    inputs="Some user input to check.",
    protect_rules=rules,
    timeout=60,
)

print(result)