Skip to main content
result = evaluator.evaluate(
    eval_templates="embedding_similarity",
    inputs={
        "expected": "The Eiffel Tower is a famous landmark in Paris, built in 1889 for the World's Fair. It stands 324 meters tall.",
        "output": "The Eiffel Tower, located in Paris, was built in 1889 and is 324 meters high."
    },
    model_name="turing_flash"
)

print(result.eval_results[0].output)
print(result.eval_results[0].reason)
Input
Required InputDescription
expectedReference content for comparison against the model generated output
outputModel-generated output to be evaluated for embedding similarity
Output
FieldDescription
ResultReturns score, where higher score indicates stronger similarity
ReasonProvides a detailed explanation of the embedding similarity assessment

​
About Embedding Similarity

It evaluates how similar two texts are in meaning by comparing their vector embeddings using distance-based similarity measures. Traditional metrics like BLEU or ROUGE rely on word overlap and can fail when the generated output is a valid paraphrase with no lexical match.

​
How Similarity Is Calculated?

Once both texts are encoded into a high-dimensional vector representations, the similarity between the two vectors u and v is computed using one of the following methods:
  1. Cosine Similarity: Measures the cosine of the angle between vectors.
Cosine Similarity=1−u⋅v∥u∥∥v∥\text{Cosine Similarity} = 1 - \frac{\mathbf{u} \cdot \mathbf{v}}{\|\mathbf{u}\| \|\mathbf{v}\|}
  1. Euclidean Distance: Measures the straight-line distance between vectors (L2 Norm). Euclidean Distance=∑i=1n(ui−vi)2\text{Euclidean Distance} = \sqrt{ \sum_{i=1}^{n} (u_i - v_i)^2 }
  2. Manhattan Distance: Measures sum of absolute differences between vectors (L1 Norm).
Manhattan Distance=∑i=1n∣ui−vi∣\text{Manhattan Distance} = {\sum_{i=1}^{n} |u_i - v_i|}
⌘I