Future AGI provides a wide range of evaluation templates to choose from where you can setup your own evaluations. You can learn more about the evaluation here.

​
1. Install the SDK

pip install futureagi

​
2. Setup the API Key

export FI_API_KEY="your_api_key"
export FI_SECRET_KEY="your_secret_key"

​
3. Setup the dataset configuration and add the evaluation

from fi.datasets import Dataset

dataset = Dataset()

dataset = dataset.get_dataset_config("<your_dataset_name>")

dataset.add_evaluation(name = "evaluation_name",
                        eval_template = "eval_template_name",
                        required_keys_to_column_names= {
                            "input": "input_column_name",
                            "output": "output_column_name"
                        })