Evaluates whether a generated response semantically contains one or more reference phrases or keywords.
0.7
)1.0
(if exact match) or 0.0
(no match) depending on whether:
match_all = False
, default)match_all = True
)Click here to learn how to setup evaluation using SDK.Input & Configuration:
Parameter | Type | Description | |
---|---|---|---|
Required Inputs | response | str | Model-generated output to be evaluated |
expected_text | str or List[str] | A single phrase or list of phrases that the response is expected to semantically include | |
Optional Config | case_insensitive | bool | Whether to lowercase input texts before comparison. Default: True |
remove_punctuation | bool | Whether to strip punctuation from texts. Default: True | |
match_all | bool | If True , all phrases must be semantically present; if False , any one match is enough. Default: False | |
similarity_threshold | float | Similarity threshold for considering a match. Typical range: 0.5 –0.9 . Default: 0.7 |
Output Field | Type | Description |
---|---|---|
score | float | Returns float between 1.0 and 0.0 , closer to 1.0 if match criteria are more satisfied, or closer0.0 otherwise |
metadata | dict | Contains similarity values for each phrase, the threshold, and match logic used |
similarity_threshold
value (if your use case allows relaxed semantic matches)."match_all"= False
if partial coverage is acceptable.