factual_accuracy is being deprecated. Avoid using it for new integrations.
Use groundedness for new implementations.
result = evaluator.evaluate(
eval_templates="groundedness",
inputs={
"input": "The capital of France is Paris.",
"output": "The capital of France is Paris.",
"context": "The capital of France is Paris.",
},
model_name="turing_flash",
)
print(result.eval_results[0].output)
print(result.eval_results[0].reason)
| Input | | | |
|---|
| Required Input | Type | Description |
| output | string | The output generated by the model |
| context | string | The context provided to the model |
| Optional Input | | |
| input | string | The input provided to the model |
| Output | | |
|---|
| Field | Description |
| Result | Returns a score, where higher values indicate better factual accuracy |
| Reason | Provides a detailed explanation of the factual accuracy assessment |
What to Do When Factual Accuracy Evaluation Gives a Low Score
When factual accuracy evaluation gives a low score, it is essential to reassess the evaluation criteria to ensure they are clearly defined and aligned with the evaluation’s goals. If necessary, adjustments should be made to enhance the criteria’s comprehensiveness and relevance. Additionally, the output should be thoroughly examined for factual inaccuracies, identifying any discrepancies and refining the content to improve factual correctness.
Differentiating Factual Accuracy with Groundedness
Factual accuracy focuses on verifying the correctness of the output based on the given input and context, ensuring that the information presented is factually sound. In contrast, groundedness ensures that the response strictly adheres to the provided context, preventing the inclusion of unsupported or external information.
While factual accuracy requires input, output, and context for evaluation, groundedness only requires a response and its context.
