Fuzzy Match

Compares two texts for similarity using fuzzy matching techniques. It's useful for detecting approximate matches between expected and generated model output when exact matching might be too strict, accounting for minor differences in wording, spelling, or formatting. 

result = evaluator.evaluate(
    eval_templates="fuzzy_match",
    inputs={
        "expected": "The Eiffel Tower is a famous landmark in Paris, built in 1889 for the World's Fair. It stands 324 meters tall.",
        "output": "The Eiffel Tower, located in Paris, was built in 1889 and is 324 meters high."
    },
    model_name="turing_flash"
)

print(result.eval_results[0].output)
print(result.eval_results[0].reason)
import { Evaluator, Templates } from "@future-agi/ai-evaluation";

const evaluator = new Evaluator();

const result = await evaluator.evaluate(
  "fuzzy_match",
  {
    expected: "The Eiffel Tower is a famous landmark in Paris, built in 1889 for the World's Fair. It stands 324 meters tall.",
    output: "The Eiffel Tower, located in Paris, was built in 1889 and is 324 meters high."
  },
  {
    modelName: "turing_flash",
  }
);

console.log(result);
Input
Required InputTypeDescription
expectedstringThe expected content for comparison against the model generated output
outputstringThe output generated by the model to be evaluated for fuzzy match
Output
FieldDescription
ResultReturns a score, where higher values indicate better fuzzy match
ReasonProvides a detailed explanation of the fuzzy match assessment

What to Do When Fuzzy Match Score is Low

  • Ensure that both input texts are properly formatted and contain meaningful content
  • This evaluation works best with texts that convey similar information but might have different wording
  • For very short texts (1-2 words), results may be less reliable
  • If you need more precise matching, consider using Levenshtein Similarity instead

Comparing Fuzzy Match with Similar Evals

  • Levenshtein Similarity: Fuzzy Match uses approximate text matching, while Levenshtein Similarity provides a stricter character-by-character comparison.
  • Embedding Similarity: Fuzzy Match compares surface-level text, while Embedding Similarity compares semantic meaning.
  • Semantic List Contains: Fuzzy Match evaluates overall text similarity, while Semantic List Contains checks if specific semantic concepts are present.
  • ROUGE Score: Fuzzy Match uses approximate matching, while ROUGE Score evaluates based on n-gram overlap, especially useful for summarization.
Was this page helpful?

Questions & Discussion