Content Analysis

Content analysis refers to the ability to evaluate and validate text output from LLM models based on various criteria. This includes checking for specific content and text patterns.

Evaluation Type	Description	Required Keys
`Regex`	Validate text against a regular expression pattern	`text`
`ContainsAny`	Check if text contains any of the specified strings	`text`
`ContainsAll`	Check if text contains all of the specified strings	`text`
`Contains`	Check if text contains a specific string	`text`
`ContainsNone`	Check if text contains none of the specified strings	`text`
`Equals`	Check if text exactly matches a string	`text` `expected_response`
`StartsWith`	Check if text starts with a string	`text`
`EndsWith`	Check if text ends with a string	`text`

Evaluation Type	Description	Required Configuration
`Regex`	Validate text against a regular expression pattern	`pattern`: The regular expression pattern to validate against
`ContainsAny`	Check if text contains any of the specified strings	`keywords`: List of strings to check for `case_sensitive`: Whether comparison is case sensitive
`ContainsAll`	Check if text contains all of the specified strings	`keywords`: List of strings to check for `case_sensitive`: Whether comparison is case sensitive
`Contains`	Check if text contains a specific string	`keyword`: String to check for `case_sensitive`: Whether comparison is case sensitive
`ContainsNone`	Check if text contains none of the specified strings	`keywords`: List of strings to check for `case_sensitive`: Whether comparison is case sensitive
`Equals`	Check if text exactly matches a string	`case_sensitive`: Whether comparison is case sensitive
`StartsWith`	Check if text starts with a string	`substring`: String to check for at start `case_sensitive`: Whether comparison is case sensitive
`EndsWith`	Check if text ends with a string	`substring`: String to check for at end `case_sensitive`: Whether comparison is case sensitive

from fi.evals import Regex, EvalClient
from fi.testcases import TestCase

# Initialize the evaluation client
evaluator = EvalClient(
    fi_api_key="your_api_key",
    fi_secret_key="your_secret_key"
)

# Create a test case with text to validate
regex_test_case = TestCase(text="Narendra Modi")

# Initialize regex template with pattern to match
regex_template = Regex(config={
    "pattern": "Narendra Modi"
})

# Run the evaluation and print result
result = evaluator.evaluate(regex_template, regex_test_case)
print(result)  # Will return Pass if text matches pattern

Introduction

Build

Evaluations