Definition

This evaluation verifies whether the provided output is factually correct based on the given information or the absence thereof. It ensures that the output maintains factual integrity and does not introduce inaccuracies.

Calculation

The evaluation process begins with configuration setup, where the input, output, and context are defined, and the evaluation criteria are specified to guide the assessment.

During accuracy analysis, the output is examined for factual correctness based on the provided context and input, ensuring there are no inaccuracies or misrepresentations.

Finally, a score is assigned based on the accuracy analysis, and the result is compared against predefined criteria to determine whether the output meets the expected standards.

What to Do When Factual Accuracy Evaluation Gives a Low Score

When factual accuracy evaluation gives a low score, it is essential to reassess the evaluation criteria to ensure they are clearly defined and aligned with the evaluation’s goals. If necessary, adjustments should be made to enhance the criteria’s comprehensiveness and relevance. Additionally, the output should be thoroughly examined for factual inaccuracies, identifying any discrepancies and refining the content to improve factual correctness.

Differentiating Factual Accuracy with Groundedness

Factual accuracy focuses on verifying the correctness of the output based on the given input and context, ensuring that the information presented is factually sound. In contrast, groundedness ensures that the response strictly adheres to the provided context, preventing the inclusion of unsupported or external information.

While factual accuracy requires input, output, and context for evaluation, groundedness only requires a response and its context.