This guide will walk you through setting up an evaluation in Future AGI, allowing you to assess AI models and workflows efficiently. You can run evaluations via the Future AGI platform or using the Python SDK.

​
Access API Key

To authenticate while running evals, you will need Future AGI’s API keys, which you can get access by following below steps:

  • Go to your Future AGI dashboard

  • Click on Keys under Developer option from left column

  • Copy both, API Key and Secret Key

​
Setup Eval Client

Install the Future AGI Python SDK using below command:

pip install futureagi

Click here to read more about our docs on using Python SDK

Then initialise the evaluation client:

from fi.evals import EvalClient

evaluator = EvalClient(
    fi_api_key="your_api_key",
    fi_secret_key="your_secret_key",
    fi_base_url="https://api.futureagi.com"
)

​
4. Running Your First Eval

This section walks you through the process of running your first evaluation using the Future AGI evaluation framework. To get started, we’ll use Tone Evaluation as an example.

​
a. Using Python SDK

Define the Test Case

Create a test case containing the text input that will be evaluated for tone.

from fi.testcases import TestCase

test_case = TestCase(
    input='''
    Dear Sir, I hope this email finds you well. 
    I look forward to any insights or advice you might have 
    whenever you have a free moment.
    '''
)

Configure the Evaluation Template

For Tone Evaluation, we use the Tone Evaluation Template to analyse the sentiment and emotional tone of the input.

from fi.evals.templates import Tone

tone_eval = Tone()

Click here to read more about all the Evals provided by Future AGI

Run the Evaluation

Execute the evaluation and retrieve the results.

result = evaluator.evaluate(eval_templates=[tone_eval], inputs=[test_case])
tone_result = result.eval_results[0].data[0]

​
b. Using Web Interface

Select a Dataset

Before running an evaluation, ensure you have selected a dataset. If no dataset is available, follow the steps to Add Dataset on the Future AGI platform.

Read more about all the ways you can add dataset

Access the Evaluation Panel

  • Navigate to your dataset.
  • Click on the Evaluate button in the top-right menu.
  • This will open the evaluation configuration panel.

Create a New Evaluation

  • Click + Create Eval to configure a new evaluation.
  • If you have already configured evaluation, you can also select them from Saved Evals.

Select The Evaluation Metric

  • Choose from the available evaluation templates.
  • Use the search bar to quickly find a specific evaluation template.
  • Here, we are using Tone eval.

Configure Evaluation Settings

The Evaluation Configuration panel allows you to set key parameters. Choose the dataset column that the evaluation should analyse.

Run the Evaluation

  • Click Save and Run to start evaluating.
  • The system applies the selected evaluation template to each row in the dataset.
  • Once complete, evaluation results will be displayed.
OverviewOverview