Purpose of Numeric Difference Eval
- It evaluate the accuracy of numerical values in model-generated outputs.
- Unlike semantic or lexical metrics which can overlook numeric discrepancies,
NumericDiff
ensures that numeric correctness is measured explicitly.
Evaluation Using SDK
Click here to learn how to setup evaluation using SDK.Input & Configuration:
Parameter | Type | Description | |
---|---|---|---|
Required Inputs | response | str | Model-generated text containing the numeric prediction. |
expected_text | str | Ground-truth text with the expected numeric value. | |
Optional Config | normalized_result | bool | Whether to return a normalised score in [0, 1] or absolute difference. If False , raw absolute error is returned. Default: True . |
Output Field | Type | Description |
---|---|---|
score | float | Score between 0 and 1 if normalized_result=True , else absolute difference. Higher is better. |