Eval Definition
Numeric Difference
Extracts numeric values from generated text and compute absolute or normalised difference between numeric value in reference text.
Purpose of Numeric Difference Eval
- It evaluate the accuracy of numerical values in model-generated outputs.
- Unlike semantic or lexical metrics which can overlook numeric discrepancies,
NumericDiff
ensures that numeric correctness is measured explicitly.
Numeric Difference Eval using Future AGI’s Python SDK
Click here to learn how to setup evaluation using the Python SDK.
Input & Configuration:
Parameter | Type | Description | |
---|---|---|---|
Required Inputs | response | str | Model-generated text containing the numeric prediction. |
expected_text | str | Ground-truth text with the expected numeric value. | |
Optional Config | normalized_result | bool | Whether to return a normalised score in [0, 1] or absolute difference. If False , raw absolute error is returned. Default: True . |
Output:
Output Field | Type | Description |
---|---|---|
score | float | Score between 0 and 1 if normalized_result=True , else absolute difference. Higher is better. |
Example:
Output: