Customizing Prompts

Mutant relies on heavily optimized LLM prompts under the hood to perform behavioral analysis, mutation planning, and mutation generation. While the default prompts are tested to work exceptionally well out-of-the-box, there are times when you may want to exert strict control over how the LLM behaves.

---

Overriding the Internal Prompts

You can override Mutant's internal prompts by passing a custom prompts dictionary into the MutationConfig object.

The prompts Dictionary Keys

The engine uses five primary prompt types during execution. You can override any or all of them:

1. behavior_analysis: Analyzes the base scenario to establish a behavioral baseline.
2. mutation_planning: Decides which specific aspects of the scenario to mutate.
3. mutation_generation: Executes the rewrite, applying the requested Dimensions.
4. quality_review: Validates the output to ensure the mutations were applied correctly.
5. coverage_analysis: Generates the final behavior tagging for HTML reports.

---

Example: Customizing Mutation Generation

Here is a copy-pasteable example of how to inject a custom system prompt specifically for the generation phase. This is useful if you want to strictly enforce formatting (like returning specific JSON) or guarantee the agent maintains a specific persona during the rewrite.

from mutant.core.scenario import Scenario
from mutant.core.config import MutationConfig
from mutant.core.engine import augment
from mutant.providers.openai import OpenAIProvider

provider = OpenAIProvider(model="gpt-4o")

scenario = Scenario(
    title="Password Reset",
    description="The user forgot their password and is asking for help.",
)

# Define your custom overrides
custom_config = MutationConfig(
    count=5,
    prompts={
        "mutation_generation": (
            "You are an expert adversarial copywriter. "
            "Rewrite the following scenario based on the instructions, "
            "but YOU MUST KEEP THE OUTPUT UNDER 50 WORDS. "
            "Do not include any pleasantries."
        )
    }
)

# Pass the config to the augment pipeline
dataset = await augment(
    dataset=[scenario],
    provider=provider,
    dimensions=["emotion.angry", "language.slang"],
    config=custom_config
)

for case in dataset.cases:
    print(f"Mutated: {case.mutated}")

[!TIP] Pro-Tip: You only need to provide the keys you want to override. Mutant will fall back to its highly-optimized defaults for any keys you omit.