Future AGI provides a wide range of evaluation templates to choose from where you can setup your own evaluations. You can learn more about the evaluation here.
1. Install the SDK
2. Setup the API Key
export FI_API_KEY="your_api_key"
export FI_SECRET_KEY="your_secret_key"
3. Setup the dataset configuration and add the evaluation
from fi.datasets import Dataset
dataset = Dataset()
dataset = dataset.get_dataset_config("<your_dataset_name>")
dataset.add_evaluation(name = "evaluation_name",
eval_template = "eval_template_name",
required_keys_to_column_names= {
"input": "input_column_name",
"output": "output_column_name"
})
