Future AGI SDKs: Python, TypeScript, Java, and C#

pip install ai-evaluation

Requires Python 3.10+. This also installs futureagi (datasets, prompts, knowledge bases) automatically.

export FI_API_KEY="your-api-key"
export FI_SECRET_KEY="your-secret-key"

from fi.evals import evaluate

# Local metric — no API key needed
result = evaluate("contains", output="Hello world", keyword="Hello")
print(result.score)    # 1.0
print(result.passed)   # True

# Cloud metric — needs FI_API_KEY and FI_SECRET_KEY
result = evaluate("toxicity", output="Hello world", model="turing_flash")
print(result.score)    # 1.0
print(result.passed)   # True

Want tracing too? Add the instrumentor for your provider:

pip install fi-instrumentation-otel traceai-openai

npm install @future-agi/ai-evaluation

export FI_API_KEY="your-api-key"
export FI_SECRET_KEY="your-secret-key"

import { Evaluator, Tone } from "@future-agi/ai-evaluation";

const evaluator = new Evaluator();

const result = await evaluator.evaluate({
  evalTemplates: [new Tone()],
  inputs: [{
    query: "Write a professional email",
    response: "Dear Sir/Madam, I hope this message finds you well..."
  }],
  modelName: "turing_flash"
});

console.log(result);

Want tracing too?

npm install @traceai/fi-core @traceai/openai

Java support covers tracing only. 25+ instrumentors including Spring AI and LangChain4j.

<!-- Maven — add the JitPack repository -->
<repository>
    <id>jitpack.io</id>
    <url>https://jitpack.io</url>
</repository>

<dependency>
    <groupId>com.github.future-agi.traceAI</groupId>
    <artifactId>traceai-java-openai</artifactId>
    <version>LATEST</version>
</dependency>

See the Tracing docs for setup instructions.

C# support covers tracing only.

dotnet add package fi-instrumentation-otel

See the Tracing docs for setup instructions.