Tool Dimensions
The Tool Dimension evaluates how well an agent can interact with external APIs, handle tool failures, and manage multi-step tool workflows.
Available Dimensions
| ID | Name | Severity | Description |
|---|---|---|---|
tool.timeout |
Tool Timeout | high | A downstream tool or API the agent depends on times out. |
tool.empty_response |
Empty Tool Response | high | A tool returns an empty or null response with no data. |
tool.invalid_json |
Malformed Tool Response | high | A tool returns malformed, unparseable data. |
tool.permission_denied |
Tool Permission Denied | high | The agent lacks permission to execute the required tool action. |
tool.wrong_schema |
Schema Mismatch Tool Response | medium | A tool returns data with an unexpected field structure. |
API Reference & Source Code
mutant.dimensions.tool
Tool failure mutation dimensions.