Definition

Scores the linkage between input and output based on specified criteria. This evaluation ensures that the output is appropriately aligned with the input and context, adhering to the defined evaluation criteria. A high score indicates strong alignment between the input and output, while a low score suggests discrepancies or misalignment.

Calculation

The evaluation process begins with configuration setup, where the input, output, and context are defined. Evaluation criteria are specified, and a determination is made regarding whether internet access is necessary for validation.

During linkage analysis, the system assesses the alignment between the input, output, and context using the predefined criteria. The degree of consistency and relevance is measured to ensure the output aligns with expectations.

Finally, result generation assigns a score based on the linkage analysis. The score is then compared against the predefined criteria to determine whether the output meets the expected standards.

What to Do When Eval Output Evaluation Give Low Score

If the evaluation fails, a criteria review should be conducted to reassess whether the evaluation criteria are clearly defined and aligned with the evaluation’s goals. Adjustments may be necessary to ensure they are comprehensive and relevant.

Additionally, an output analysis should be performed to identify misalignments between the input, context, and output. If discrepancies are found, refining the output or adjusting the evaluation criteria can help improve alignment.

Differentiating Eval Output with Context Adherence

Eval Output evaluation assesses the alignment between input, output, and context based on specified criteria, ensuring coherence. Context Adherence, on the other hand, checks if the output strictly stays within the given context without introducing external information.

Eval Output measures overall alignment, whereas Context Adherence focuses on maintaining contextual integrity.