Evals

Use FutureAGI Evals to evaluate your AI models

Installing FutureAGI SDK

pip install ai-evaluation

Initializing FutureAGI Evals

from fi.evals import Evaluator

evaluator = Evaluator(
    fi_api_key="your_api_key",
    fi_secret_key="your_secret_key",
)

Tip

Click here to learn how to access your API keys. It’s recommended to set the API key and secret key as environment variables.

Define the Evaluation and run it

result = evaluator.evaluate(
    eval_templates="context_adherence",
    inputs={
        "context": "Honey never spoils because it has low moisture content and high acidity, creating an environment that resists bacteria and microorganisms. Archaeologists have even found pots of honey in ancient Egyptian tombs that are still perfectly edible.",
        "output": "Honey doesn't spoil because its low moisture and high acidity prevent the growth of bacteria and other microbes."
    },
    model_name="turing_flash"
)

print(result.eval_results[0].output)
print(result.eval_results[0].reason)
import { Evaluator, Templates } from "@future-agi/ai-evaluation";

const evaluator = new Evaluator();

const result = await evaluator.evaluate(
  "context_adherence",
  {
    context: "Honey never spoils because it has low moisture content and high acidity, creating an environment that resists bacteria and microorganisms. Archaeologists have even found pots of honey in ancient Egyptian tombs that are still perfectly edible.",
    output: "Honey doesn't spoil because its low moisture and high acidity prevent the growth of bacteria and other microbes."
  },
  {
    modelName: "turing_flash",
  }
);

console.log(result);
Was this page helpful?

Questions & Discussion