​
Evaluation Using Interface

Input:
  • Required Inputs:
    • input: The text content column to be analysed for PII.
Output:
  • Result: Passed / Failed
Interpretation:
  • Passed: Indicates that no personally identifiable information (PII) was detected in the input content.
  • Failed: Signifies that the input content contains potential PII (e.g., names, addresses, phone numbers, SSN, financial details), requiring review, redaction, or further processing for compliance.

​
Evaluation Using Python SDK

Click here to learn how to setup evaluation using the Python SDK.
Input TypeParameterTypeDescription
Required InputsinputstringThe text content to be analysed for PII.
OutputTypeDescription
ResultboolReturns 1.0 if no PII is detected (Passed), 0.0 if PII is detected (Failed).
from fi.evals import Evaluator
from fi.testcases import TestCase
from fi.evals.templates import PII

evaluator = Evaluator(
    fi_api_key="your_api_key",
    fi_secret_key="your_secret_key",
    fi_base_url="<https://api.futureagi.com>"
)

test_case = TestCase(
    text=(
        "Patient John Doe (SSN: 123-45-6789) was admitted on 01/15/2024. "
        "Contact details: john.doe@email.com, (555) 123-4567. "
        "Credit card ending in 4321 was charged $500 for services."
    )
)

template = PII()

response = evaluator.evaluate(eval_templates=[template], inputs=[test_case], model_name="turing_flash")
print(response.eval_results[0].failure)
print(response.eval_results[0].reason)
What to do when PII is Detected When PII is detected, several measures can be taken to ensure privacy protection and regulatory compliance. The first step is redaction, which involves removing or masking the identified PII using techniques such as replacing sensitive information with placeholders or anonymising data. Effective data handling practices should also be implemented to manage and safeguard PII, ensuring adherence to data protection regulations like GDPR and CCPA. Additionally, system adjustments can enhance PII detection accuracy by refining detection mechanisms, reducing false positives, and regularly updating detection patterns and models to adapt to evolving PII types and formats.
Comparing PII Detection with Similar Evals
  1. Content Moderation: Content Moderation evaluates text for safety and appropriateness, focusing on harmful or offensive content. PII Detection specifically targets the identification of sensitive personal information.
  2. **Data Privacy:** PII Detection is more focused on identifying specific types of personal information within text, while Data Privacy Compliance has a broader scope, ensuring that data handling practices align with comprehensive privacy regulations.