LLM Providers

Mutant is completely provider agnostic. You are not locked into any single ecosystem. Whether you want to use state-of-the-art commercial models via OpenAI/Anthropic, or you want to run completely local open-source models for privacy and cost savings via Ollama, Mutant supports it.

Under the hood, all provider classes inherit from BaseLLMProvider.

---

Built-in Providers

Mutant ships with built-in integrations for the most popular LLM providers.

1. OpenAI

Uses the official OpenAI SDK. Ideal for GPT-4o, GPT-4-turbo, etc.

from mutant.providers.openai import OpenAIProvider

provider = OpenAIProvider(
    model="gpt-4o",
    api_key="sk-...", # Optional: Defaults to os.environ["OPENAI_API_KEY"]
)

2. Ollama (Local & Open-Source)

Uses the Ollama API to run models completely locally. This is highly recommended for privacy-sensitive red teaming or high-volume dataset generation.

from mutant.providers.ollama import OllamaProvider

provider = OllamaProvider(
    model="llama3.1",
    base_url="http://localhost:11434" # Optional: Defaults to localhost
)

3. Anthropic

Uses the official Anthropic SDK for the Claude 3 model family.

from mutant.providers.anthropic import AnthropicProvider

provider = AnthropicProvider(
    model="claude-3-5-sonnet-20240620",
    api_key="sk-ant-..." # Optional: Defaults to os.environ["ANTHROPIC_API_KEY"]
)

4. LiteLLM (Universal Router)

If you need to connect to Azure, AWS Bedrock, Cohere, Vertex AI, or any of the 100+ providers supported by LiteLLM, you can use the LiteLLMProvider.

from mutant.providers.litellm import LiteLLMProvider

provider = LiteLLMProvider(
    model="azure/gpt-4o",
    api_key="...",
    api_base="https://my-endpoint.openai.azure.com/"
)

---

Provider Resilience (Retries & Reliability)

Because LLMs can be flaky (network timeouts, rate limits, or outputting malformed JSON), every built-in Provider automatically implements retry logic.

By default, providers will attempt an operation 3 times before failing. You can customize this globally per-provider:

provider = OllamaProvider(
    model="llama3.1",
    max_retries=5  # Try 5 times before failing
)

---

Creating a Custom Provider

If you have a proprietary internal model endpoint, you can easily plug it into Mutant by subclassing BaseLLMProvider.

You only need to implement one method: generate_completion().

from mutant.providers.base import BaseLLMProvider

class MyCustomProvider(BaseLLMProvider):
    def __init__(self, model: str):
        super().__init__(model=model)

    async def generate_completion(self, messages: list[dict], temperature: float = 0.7, max_tokens: int = 2000) -> str:
        # Implement your custom API call here.
        # `messages` is a standard list of {"role": "...", "content": "..."} dicts.

        response_text = await my_custom_api_call(
            messages=messages,
            temperature=temperature
        )

        return response_text

Once defined, you can pass MyCustomProvider directly into augment() or red_team() just like any built-in provider!