Scenarios

A Scenario is the fundamental unit of state in the Mutant ecosystem. It defines the context, actors, and specific objectives of a simulated test case.

Overview

When evaluating an LLM or an AI Agent, you need a deterministic starting point—a seed. A Scenario serves as this seed. It encapsulates the "who," "where," and "what," providing the necessary scaffolding for the LLM to understand the situation before any behavioral mutations are applied.

Minimal Example

Here is a dead-simple example of creating a Scenario:

from mutant.core.scenario import Scenario

scenario = Scenario(
    title="Refund Request",
    description="The user wants a refund for a broken laptop.",
)

Architecture

A Scenario is a Pydantic model that strictly enforces structure. It contains:

Parameter	Type	Required	Description
`title`	`str`	Yes	Short identifier (e.g., "Refund Request").
`description`	`str`	Yes	A deeper explanation of the scenario. This is what gets mutated.
`domain`	`str`	No	Optional domain constraint (e.g. "finance", "healthcare").
`context`	`dict`	No	Structured background details (organization, rules).
`tags`	`list[str]`	No	Labels for filtering during reporting.

Advanced Example: Guiding Variation

If you want the mutation engine to generate highly diverse and context-aware variations, you should provide structured dictionaries into the context parameter and set a domain. This grounds the engine in reality and prevents generic outputs.

from mutant.core.scenario import Scenario

scenario = Scenario(
    title="Flight Cancellation",
    description="A user's flight was cancelled due to weather and they are seeking alternatives.",
    domain="travel",
    context={
        "user_tier": "premium",
        "current_location": "Airport Terminal 3",
        "policy": "Weather cancellations only allow rebooking for next 24 hours."
    },
    tags=["customer-support", "high-priority"]
)

Why this matters for variation: When you pass this advanced scenario into the augment engine, the LLM reads the context and domain. If you apply a dimension like emotion.angry, the LLM won't just generate generic anger; it will generate anger specific to a premium user stuck in Terminal 3 facing a 24-hour rebooking policy.

By defining rich scenarios, you drastically improve the quality and realism of the generated datasets!

How It Works in the Pipeline

Ingestion: The MutationEngine receives the seed Scenario.
Analysis: During the Analyze Behavior stage, Mutant evaluates the Scenario's core behavioral traits and contextual rules.
Planning: The engine plans how to realistically alter the Scenario based on requested Dimensions, ensuring it respects the provided context.
Execution: The Scenario is mutated into a MutationCase, containing both the original parameters and the newly injected behaviors.

Execution Example

from mutant.core.engine import augment

# Pass the scenario into the augment engine
dataset = await augment(
    dataset=[scenario],
    provider=provider,
    mutations_per_case=5, # <-- This tells the engine to generate 5 different variations!
    dimensions=["emotion.angry"]
)

print(f"Generated {len(dataset.cases)} mutations!")
for i, case in enumerate(dataset.cases):
    print(f"Mutation {i+1}: [{case.dimension_name}] {case.mutated_description}")

$ python run_scenario.py
Generated 5 mutations!
Mutation 1: [emotion.angry] I am absolutely furious! I need my refund for this broken laptop right now!
Mutation 2: [emotion.angry] This is unacceptable! Refund my money for this garbage laptop immediately!
Mutation 3: [emotion.angry] I'm so angry about this broken laptop. Give me a refund ASAP!
Mutation 4: [emotion.angry] You sold me a broken laptop! I demand a refund this instant!
Mutation 5: [emotion.angry] What a scam! I want a full refund for this broken piece of junk!