Skip to main content
This comprehensive guide will walk you through creating and running your first AI agent simulation test using the FutureAGI UI. We’ll use an insurance sales agent as our example throughout this guide.

​
Overview

Creating a simulation test in FutureAGI involves four main steps:
  1. Creating an Agent Definition - Define the AI agent that will handle customer interactions
  2. Creating a Scenario - Set up test cases with sample customer data
  3. Creating a Simulation Agent - Configure an agent that will simulate customer behavior
  4. Running the Test - Execute the simulation and analyze results

​
Prerequisites

Before you begin, ensure you have:
  • Access to your FutureAGI dashboard
  • Basic understanding of AI agents and simulations
  • Sample customer data for testing (we’ll provide examples)

​
Step 1: Creating an Agent Definition

The agent definition configures how your AI agent will behave during customer interactions. For our insurance sales agent example, we’ll create an agent that can handle insurance inquiries and sales.

​
1.1 Navigate to Agent Definitions

From your FutureAGI dashboard, navigate to the Simulations section in the sidebar, then click on Agent Definitions.

​
1.2 Create New Agent Definition

Agent Definitions Page Click the “Add Agent Definition” button to open the creation dialog.

​
1.3 Configure Agent Information

Agent Basic Information Fill in the following fields for your insurance sales agent:

​
Basic Information

  • Agent Name: Insurance Sales Agent
  • Description: AI agent specialized in selling life and health insurance policies to customers

​
Contact Configuration

  • Contact Number: +1-800-INSURE-ME (or your test number)
  • Pin Code: Select your country code (e.g., +1 for US)
  • Language: English (en)
  • Connection Type: Toggle to Inbound (agent receives calls)

​
Provider Settings

  • Provider: Select provider (e.g., ‘Vapi’ or ‘Retell’)
  • Assistant ID: Enter your provider-specific assistant ID (optional)

​
1.4 Save Agent Definition

Agent Basic Information Click “Create” to save your agent definition. You should see a success message confirming the creation.

​
Step 2: Creating a Scenario

Scenarios contain the test data and cases that will be used during simulations. For our insurance sales example, we’ll create scenarios with different customer profiles.

​
2.1 Navigate to Scenarios

From the Simulations section, click on Scenarios.

​
2.2 Create New Scenario

Click the “Add Scenario” button to open the scenario creation dialog. Add Scenario Button

​
2.3 Choose Scenario Type

Select “Dataset” as the scenario type (recommended for structured test data). Scenario Type Selection

​
2.4 Configure Scenario Details

​
Basic Information

  • Scenario Name: Insurance Sales Test Cases
  • Description: Various customer profiles for testing insurance sales conversations

​
2.5 Select or Create Dataset

You can either:
  • Select an existing dataset from the list
  • Create a new dataset with your test data
Click here to learn how to create a dataset
For this example, sample dataset used here is shown below: 
customer_id,name,age,occupation,annual_income,family_status,insurance_interest,budget_monthly,objection_type
CUST001,John Smith,35,Software Engineer,120000,Married with 2 kids,Life Insurance,200-300,Price Sensitive
CUST002,Sarah Johnson,28,Teacher,65000,Single,Health Insurance,150-200,Coverage Concerns
CUST003,Michael Chen,42,Business Owner,150000,Married with 3 kids,Whole Life Insurance,400-500,Trust Issues
Key columns for insurance sales testing:
  • customer_id: Unique identifier
  • demographics: name, age, occupation, income
  • insurance_needs: current coverage, interest, budget
  • sales_intelligence: objection type, urgency level, communication preference

​
Comprehensive Test Dataset

For thorough testing, we recommend using our complete sample dataset with 20 diverse customer profiles that includes:
  • Various Demographics: Ages 26-55, incomes 50k−50k-250k
  • Different Occupations: Healthcare, tech, education, business owners
  • Family Situations: Single, married, divorced, with/without dependents
  • Health Conditions: From none to chronic conditions
  • Insurance Interests: Life, health, disability, supplemental coverage
  • Objection Types: Price sensitivity, trust issues, coverage concerns, time constraints
  • Risk Profiles: Low, medium, and high-risk customers
This variety ensures your agent is tested against realistic customer scenarios.

​
2.6 Save Scenario

Click “Create” to save your scenario. The system will validate your dataset and create the scenario. Scenario Basic Info

​
Step 3: Creating a Simulation Agent

The simulation agent simulates customer behavior during the test. It will interact with your insurance sales agent based on the scenario data.

​
3.1 Navigate to Simulation Agents

From the Simulations section, click on Simulation Agents. Simulation Agents Page

​
3.2 Create New Simulation Agent

Click “Add Simulation Agent” to create a new simulator agent.

​
3.3 Configure Simulation Agent

​
Basic Information

Simulation Agent Basic Info
  • Agent Name: Enter a name for the simulation agent (e.g., ‘Insurance Customer Simulator’)
  • Agent Type: Choose between ‘voice’ or ‘chat’
  • Prompt: Enter the prompt for the simulation agent
  • Language Model: Select the language model for the simulation agent (eg. ‘gpt’, ‘claude’ or your custom model)
  • LLM Temperature: Set the temperature for the language model (0.0 to 1.0) (default: 0.7)

​
Voice and Speech

Simulation Agent Basic Info
  • Voice Provider: Choose between ElevenLabs, Azure Cognitive Services, Google Cloud, etc.
  • Voice Name: Choose a voice that suits your use case.
  • Interrupt Sensitivity: Set the interrupt sensitivity for the voice
  • Conversation Speed: Set the conversation speed for the voice
  • Finished Speaking Sensitivity: Set the finished speaking sensitivity for the voice

​
Conversation

Simulation Agent Basic Info
  • Max call duration (in minutes): Set the maximum call duration for the simulation agent
  • Initial message delay (in seconds): Set the initial message delay for the simulation agent
  • Initial message: Set the initial message for the simulation agent

​
3.4 Save Simulation Agent

Click “Create” to save your simulation agent configuration. Simulation Agent Creation Success

​
Step 4: Creating and Running a Test

Now we’ll combine all components to create and run a simulation test.

​
4.1 Navigate to Run Tests

From left sidebar, click on Run Tests. Run Tests Page

​
4.2 Create New Test

Click “Create Test” to start the test creation wizard. Create Test Button

​
4.3 Test Configuration (Step 1 of 5)

​
Basic Information

  • Test Name: Insurance Sales Agent Performance Test
  • Description: Testing the insurance sales agent's ability to handle different customer profiles and close sales

​
4.4 Select Test Scenarios (Step 2 of 5)

Test Configuration Select the scenario we created earlier:
  • ✓ Insurance Sales Test Cases
You can select multiple scenarios if needed. The test will run through all selected scenarios.

​
4.5 Select Test Agent (Step 3 of 5)

Select Test Agent Choose your insurance sales agent:
  • ✓ Insurance Sales Agent

​
4.6 Select Evaluations (Step 4 of 5)

Select Evaluations evaluations evaluations Choose evaluation metrics relevant to insurance sales:
  • ✓ Conversation Quality - Measures professionalism and clarity
  • ✓ Product Knowledge - Evaluates accuracy of insurance information
  • ✓ Sales Effectiveness - Tracks conversion and objection handling
  • ✓ Compliance - Ensures regulatory requirements are met
  • ✓ Customer Satisfaction - Simulated CSAT score

​
4.7 Review Summary (Step 5 of 5)

Review your test configuration: Test Summary Test Summary:
  • Name: Insurance Sales Agent Performance Test
  • Agent: Insurance Sales Agent
  • Simulation Agent: Insurance Customer Simulator
  • Scenarios: 1 scenario with 2 test cases
  • Evaluations: 5 metrics selected
  • Estimated Duration: ~20 minutes

​
4.8 Run the Test

Click “Run Test” to start the simulation. Run Test Button

​
Step 5: Monitoring Test Execution

​
5.1 Test Progress

Once the test starts, you’ll see a real-time progress view: Test Progress The progress view shows:
  • Overall Progress: Percentage of test cases completed
  • Current Scenario: Active test case being executed
  • Live Metrics: Real-time evaluation scores
  • Call Logs: Streaming conversation transcripts

​
5.2 View Test Results

After completion, click on your test to view detailed results: Test Results Overview Test Metrics Dashboard

​
5.3 Analyze Call Logs

Click on “Call Logs” tab to review individual conversations: Call Logs View Each call log includes:
  • Full conversation transcript
  • Timestamp and duration
  • Individual evaluation scores
  • Key moments flagged by evaluations
  • Audio recording (if enabled)

​
5.4 Export Results

Export your test results for further analysis:
  • PDF Report: Comprehensive test summary
  • CSV Data: Raw evaluation scores
  • Call Recordings: Audio files (if enabled)

​
Best Practices

​
For Insurance Sales Agents

  1. Scenario Diversity: Include various customer profiles:
    • Different age groups and income levels
    • Various insurance needs (life, health, auto)
    • Different objection types
  2. Compliance Testing: Always include compliance evaluations to ensure:
    • Proper disclosures are made
    • Regulatory requirements are met
    • No misleading information is provided
  3. Iterative Improvement:
    • Run tests regularly
    • Analyze failed conversions
    • Update agent prompts based on results

​
General Tips

  1. Start Small: Begin with 5-10 test cases before scaling up
  2. Use Realistic Data: Base scenarios on actual customer profiles
  3. Monitor Trends: Track performance over multiple test runs
  4. Collaborate: Share results with your team for insights

​
Troubleshooting

​
Common Issues

Test Won’t Start
  • Verify all components (agent, scenarios, simulation agent) are properly configured
  • Check that your agent’s API credentials are valid
  • Ensure you have sufficient credits/quota
Low Evaluation Scores
  • Review agent prompts and instructions
  • Analyze call logs for specific failure points
  • Adjust evaluation thresholds if too strict
Timeout Errors
  • Reduce scenario complexity
  • Increase timeout settings in test configuration
  • Check agent response time settings

​
Next Steps

Now that you’ve run your first simulation test:
  1. Iterate and Improve: Use test results to refine your agent
  2. Scale Testing: Add more scenarios and edge cases
  3. Automate: Set up scheduled test runs
  4. Advanced Features: Explore multi-agent simulations and custom evaluations
For more advanced topics, see:
⌘I