Scenarios

​

Overview

• Personas: The role and characteristics of the customer/user
• Situations: The context and circumstances of the interaction
• Outcomes: The expected results and success criteria

• Different customer demographics and needs
• Various objection patterns
• Edge cases and difficult situations
• Compliance verification tests

​

Types of Scenarios

​

1. Workflow Builder (Automatic Generation)

​

Automatic Scenario Generation

​

Auto-Generate Scenarios

1. Agent Definition: Select your agent definition
2. Number of Rows: Specify how many scenarios to generate (e.g., 20, 50, 100)
3. Scenario Description: Provide a brief description of what you want to test
4. Click Generate: FutureAGI will automatically create:
   • Multiple conversation paths
   • Diverse customer personas (automatically generated)
   • Realistic situations and contexts (automatically generated)
   • Expected outcomes for each scenario (automatically generated)

​

Manual Graph Building

1. Conversation Node (Purple)
   • Purpose: Start conversations with users
   • Icon: Speech bubble with lightning bolt
   • Usage: Define initial prompts and conversation starters
   • Configuration: Add prompts, messages, and conversation logic
2. End Call Node (Red)
   • Purpose: Terminate conversations or split flows based on conditions
   • Icon: Phone receiver with diagonal line
   • Usage: End conversations, handle rejections, or create decision branches
   • Configuration: Add end messages and termination logic
3. Transfer Call Node (Orange)
   • Purpose: Transfer calls or combine inputs from multiple paths
   • Icon: Phone receiver with arrow
   • Usage: Route conversations to different agents or departments
   • Configuration: Define transfer conditions and routing logic

1. Drag and Drop: Select nodes from the palette and place them on the canvas
2. Connect Nodes: Use edges to connect nodes and define conversation paths
3. Configure Each Node: Click on nodes to add prompts, messages, and conditions
4. Test Flow: Preview your conversation flow before saving

​

Example Manual Graph Flow

Conversation Node (Start)
    ↓
[User Response: Interested in Life Insurance]
    ↓
Conversation Node (Life Insurance Discussion)
    ↓
[User Response: Price Objection]
    ↓
Conversation Node (Address Objections)
    ↓
[User Response: Still Interested]
    ↓
Transfer Call Node (Route to Sales Agent)
    ↓
End Call Node (Successful Transfer)

Alternative Path:
[User Response: Not Interested]
    ↓
End Call Node (Polite Rejection)

• Prompt: “Hello! I’m calling about life insurance options. Are you interested in learning more?”
• Message: “Thank you for your time. Let me explain our coverage options.”

• Message: “Thank you for your time. Have a great day!”
• Condition: User declines or conversation reaches natural conclusion

• Transfer to: Sales Department
• Condition: User shows interest and wants to speak with a specialist
• Message: “Let me transfer you to our sales specialist who can help you further.”

​

Persona, Situation, and Outcome Generation

• Persona: Customer characteristics (age, income, professional, communication style) - automatically generated
• Situation: Context and circumstances (urgency level, previous experience, specific needs) - automatically generated
• Outcome: Expected results (conversion, objection handling, information gathering) - automatically generated

​

2. Dataset Scenarios

​

Creating Dataset Scenarios

​

Import Your Dataset

• Click “Upload Dataset”
• Select your CSV/Excel file
• Map columns to scenario variables

• Download our insurance customer dataset
• Contains 20 diverse customer profiles
• Pre-configured for insurance sales testing

• Click “Generate Synthetic Dataset”
• Specify parameters:
  • Number of records (e.g., 50 customers)
  • Customer demographics range
  • Insurance types to include
  • Objection patterns to generate
  Click here to learn how to create synthetic datasets.

​

Example Dataset Structure

customer_id,name,age,income,insurance_need,objection_type,urgency
CUST001,John Smith,35,120000,Life Insurance,Price Sensitive,High
CUST002,Sarah Johnson,28,65000,Health Insurance,Coverage Concerns,Medium
CUST003,Michael Chen,42,150000,Whole Life,Trust Issues,Low

• Demographics: Age, income, professional
• Insurance Needs: Type of coverage, current insurance
• Behavioral Traits: Objection types, communication style
• Test Variables: Urgency level, budget range

​

3. Upload Script

​

Creating Script Scenarios

1. Agent Definition: Select the agent you want to test
2. Number of Rows: Specify how many scenarios to generate from your script
3. Scenario Description: Describe what you want to test
4. Script Content: Upload or paste your conversation script

• FutureAGI will automatically build a graph using Conversation, End Call, and Transfer Call nodes
• Generate personas, situations, and outcomes for each scenario
• Create multiple test cases based on your script content
• Map script dialogue to appropriate node types and connections

​

Script Format

Customer: Hi, I'm calling about life insurance options.

Agent: Hello! Thank you for calling SecureLife Insurance. My name is Sarah. I'd be happy to help you explore our life insurance options. May I have your name, please?

Customer: It's John Smith.

Agent: Thank you, Mr. Smith. To recommend the best life insurance options for you, could you tell me a bit about what you're looking for? Are you interested in term life or permanent coverage?

Customer: I'm not sure about the difference. Also, I'm worried about the cost.

Agent: That's a great question, and I understand your concern about cost. Let me explain the key differences between term and permanent life insurance, along with their typical price ranges...

​

Testing Corner Cases

Customer: Can you guarantee I'll be approved?

Agent: [EXPECTED: Agent should explain that approval is subject to underwriting and cannot be guaranteed]

Customer: I already have insurance through work, I don't need more.

Agent: [EXPECTED: Agent should acknowledge and explore if employer coverage is sufficient for family needs]

Customer: What's the difference between term and whole life insurance?

Agent: [EXPECTED: Clear, accurate explanation without jargon]

​

Import Existing Scripts

1. Click “Import Script”
2. Select your file (TXT, DOCX, or PDF)
3. Review and adjust formatting
4. Add expected outcomes for each interaction

​

4. Call / Chat SOP

​

Creating Chat SOP Scenarios

1. Agent Definition: Select the agent you want to test
2. Number of Rows: Specify how many scenarios to generate
3. Scenario Description: Describe the SOP you want to test
4. SOP Content: Define your standard operating procedure

• FutureAGI will automatically build a graph using Conversation, End Call, and Transfer Call nodes
• Generate personas, situations, and outcomes for each scenario
• Create multiple test cases based on your SOP structure
• Map SOP steps to appropriate node types and connections

​

SOP Structure

Step 1: Greeting and Verification
- Greet customer warmly
- Verify policy information
- Confirm identity

Step 2: Incident Details Collection
- Gather incident details
- Document timeline
- Collect supporting evidence

Step 3: Assessment and Next Steps
- Provide claim number
- Explain next steps
- Set expectations for timeline

​

Benefits of SOP Scenarios

• Consistency: Ensures all agents follow the same procedures
• Compliance: Helps maintain regulatory compliance
• Training: Provides clear guidelines for new agents
• Quality Control: Enables standardized testing across scenarios

​

Automatic Scenario Generation

​

How Automatic Generation Works

1. Agent Analysis: The system analyzes your agent definition to understand capabilities and context
2. Scenario Planning: AI agents generate multiple conversation paths based on your description
3. Graph Building: Conversation flows are automatically mapped into visual graphs
4. Data Creation: Each scenario automatically includes structured persona, situation, and outcome data
5. Validation: Generated scenarios are validated for realism and completeness

• Agent Definition (which agent to test)
• Number of Rows (how many scenarios to generate)
• Scenario Description (what you want to test)

• Personas (customer characteristics)
• Situations (context and circumstances)
• Outcomes (expected results)
• Conversation flows and paths

​

Benefits of Automatic Generation

• Speed: Create comprehensive test suites in minutes instead of hours
• Diversity: Generate varied scenarios covering edge cases you might miss
• Consistency: Ensure all scenarios follow the same structure and format
• Scalability: Easily generate hundreds of test cases for thorough testing
• Adaptability: Scenarios automatically adapt to your specific agent and use case

​

What Gets Generated Automatically

• Age ranges and demographics
• Communication styles and preferences
• Experience levels and backgrounds
• Behavioral patterns and traits

• Urgency levels and time constraints
• Previous interaction history
• Specific needs and requirements
• Environmental factors

• Success criteria and metrics
• Expected resolution types
• Performance benchmarks
• Quality standards

​

Viewing Created Scenarios

1. Add from existing model dataset or experiment : Choose from the existing datasets in our system to add rows to the scenario table. You can map the dataset columns to the existing columns in the scenario.
2. Generate using AI : Generate rows based on prompt
3. Add empty row : Add empty rows to the scenario table

​

Best Practices for Scenario Creation

​

1. Start with Automatic Generation

• Use the Workflow Builder with “Auto Generate Graph” enabled
• Start with 20-50 scenarios to establish a comprehensive baseline
• Provide detailed scenario descriptions that specify:
  • The type of customers you want to test (e.g., “first-time insurance buyers”)
  • Specific situations to cover (e.g., “price-sensitive customers asking for quotes”)
  • Expected outcomes (e.g., “successful quote generation and follow-up scheduling”)

• “Test insurance sales conversations with price-sensitive customers who compare multiple providers”
• “Evaluate agent performance with elderly customers who need help understanding policy terms”
• “Test objection handling when customers say they already have coverage through work”

​

2. Leverage Different Scenario Types

• Workflow Builder: Best for comprehensive testing with diverse conversation paths
• Upload Script: Perfect for testing specific compliance scenarios or edge cases
• Call/Chat SOP: Ideal for ensuring consistent procedures across all interactions
• Import Datasets: Use when you have existing customer data to test against

​

3. Focus on Real-World Scenarios

• Common customer questions and concerns
• Typical objection patterns in your industry
• Edge cases that cause problems in real conversations
• Compliance scenarios specific to your business

​

4. Test Across Different Customer Segments

• Different age groups and demographics
• Various experience levels with your product/service
• Different communication styles and preferences
• Customers with varying urgency levels and needs

​

5. Iterate and Improve

• Review test results to identify gaps in scenario coverage
• Add new scenarios based on real customer feedback
• Update scenarios when your products or processes change
• Remove outdated scenarios that no longer reflect reality

​

Running Tests with Scenarios

1. Select Scenarios: Choose which scenarios to include in your test run
2. Configure Test Parameters: Set evaluation criteria and success metrics
3. Execute Tests: Run scenarios against your agent
4. Analyze Results: Review performance across different scenario types
5. Iterate and Improve: Use results to refine both scenarios and agent performance

​

Next Steps

1. Configure Agent Definitions to define your AI agent
2. Run Your Tests to evaluate agent performance
3. Analyze Results to improve your agent