Completeness

Definition

Completeness evaluates whether the response fully addresses the input query. This evaluation is crucial for ensuring that the generated response is comprehensive and leaves no aspect of the query unanswered.

Calculation

It begins with an analysis of the input query to understand its requirements and expectations, followed by an examination of the output response to identify the information provided.

An assessment is then conducted to determine whether all aspects of the query have been addressed, ensuring the response meets the expected completeness criteria. A gap analysis follows, identifying any missing elements or incomplete sections that may impact the response quality.

Eval return a numerical score ranging from 0 to 10 is assigned, quantifying how well the response covers the query and providing a clear measure of its adequacy.

What to do when Completeness is Low

Determine which aspects of the query have not been fully addressed and identify any gaps or incomplete sections that require additional information.

Enhancing the response involves adding missing details to ensure it is comprehensive and refining the content to cover all aspects of the query.

To improve completeness in the long term, implementing mechanisms that align responses more closely with query requirements and enhancing the response generation process to prioritise completeness can help ensure more thorough and accurate outputs.

Differentiating Completeness with Answer Relevancy

While Completeness focuses on whether all aspects of a query are addressed, Answer Relevancy determines how well the response aligns with the query’s intent. A response can be complete yet irrelevant, or relevant but incomplete.

Completeness evaluates the thoroughness of coverage, whereas Answer Relevancy assesses whether the response appropriately addresses the query’s core purpose.