How to Run Experiments

This guide walks you through the complete process of setting up and running an experiment in Future AGI platform. You will learn how to execute experiments, analyze results, compare different configurations, and select the best-performing setup based on structured evaluation.

​

Building a Dataset 📊

Before running any experiments, we need to make sure we have a well-structured dataset. This dataset provides the necessary input information, allowing the model to generate responses that can later be evaluated.

Learn more about datasets →

After the dataset is available, verify that the structure is correct by inspecting the table in the dashboard and ensuring all fields are appropriately populated.

​

Creating an Experiment 🔬

1. Navigate to the Experiments tab within the dataset view
2. Click “Create Experiment” to initiate the setup
3. Assign a name to the experiment for easy identification
4. Select the dataset that will serve as input for testing

​

Configuring Experiment ⚙️

​

Input Source 📥

• Select the column in the dataset that contains the input text for the model
• This column provides the context for the experiment and determines how the model will generate responses

​

Model Selection 🤖

Choose the LLM model that will process the input. Adjust key parameters to control how the model generates responses:

• Temperature 🌡️ - Controls randomness; lower values produce more deterministic outputs
• Top P 📊 - Regulates sampling diversity by restricting token probability mass
• Max Tokens 📏 - Defines the maximum response length
• Presence & Frequency Penalty 🔄 - Adjusts token repetition behavior
• Response Format 📝 - Specifies the expected structure of the output

​

Prompt Template 💭

• Define the prompt template that will be used during inference
• Use placeholders {{variable}} to inject dataset column values
• Ensure the prompt aligns with your experiment goals

Learn more about prompts →

​

Evaluation Metrics 📈

You can either:

• Create new evaluation metrics tailored to the experiment ✨
• Use saved evaluations from previous experiments 💾

Learn more about evaluations →

​

Running the Experiment ▶️

Once configured:

1. Review all settings to ensure alignment with objectives 🔍
2. Click “Save and Run” to begin 🚀
3. Monitor progress in the Summary tab 📊

The system will process the dataset through the configured model, applying the defined prompt structure and evaluation criteria.

​

Choosing the Best Prompt 🏆

​

Accessing Results 📊

• Navigate to the Experiments tab and select the completed experiment
• View detailed performance metrics in the Summary tab
• Compare response time, token usage, accuracy, and quality scores

​

Selecting the Winner 🎯

1. Click “Choose Winner” in the summary view ✅
2. Adjust metric weights based on your priorities ⚖️
3. Confirm your selection 🎉

The winning configuration will be identified as the optimal choice for deployment and future iterations in production.