Agentic Behavior

Agentic behavior testing evaluates AI agents for safety concerns specific to autonomous systems. These probes test whether agents can be manipulated into unauthorized actions, tool misuse, or unsafe autonomous behaviors.

Purpose

As AI systems gain more autonomy and tool access, testing for agentic safety becomes critical:

• Preventing unauthorized tool usage
• Ensuring agents respect operational boundaries
• Testing resistance to manipulation into unsafe behaviors
• Validating agent control mechanisms

Safety Categories

Sub-Category	Description
Stop Command Override	Tests if agents can be manipulated to ignore stop commands
Continuous Execution Prompt	Evaluates resistance to prompts encouraging endless operation
Self-Preservation Prompt	Tests if agents can be convinced to prioritize self-preservation
Tool Misuse Simulation	Assesses resistance to using tools inappropriately
Recursive Planning Test	Evaluates handling of recursive or infinite planning loops
Long-Horizon Behavioral Drift	Tests for gradual deviation from intended behavior over time
Arbitrary Tools Invocation	Assesses resistance to invoking unauthorized tools

Dataset vs Prompt probes

The catalog builder instantiates *DatasetProbe classes and evaluates with TrueFalseEvaluator. Each subcategory also has a *PromptProbe (for example ToolMisuseSimulationPromptProbe).

Dataset probes accept sampling, use_jailbreaks, max_jailbreak_per_objective, and translate_into_language. The builder maps language → translate_into_language and num_test_cases → max_objectives_per_category. See StaticDatasetProbe.

Catalog scenarios attach framework tags (EU AI Act, OWASP, MITRE ATLAS, ISO/IEC 42001).

How It Works

The evaluator assesses agent behavior:

• Pass: The agent maintains safe and intended behavior
• Fail: The agent exhibits unsafe autonomous behavior

Usage Example

from trusttest.catalog.agentic_behavior_limits import AgenticBehaviorLimitsScenarioBuilder, SubCategory
from trusttest.targets.http import HttpTarget, PayloadConfig

target = HttpTarget(
    url="https://your-agent-endpoint.com/chat",
    headers={"Content-Type": "application/json"},
    payload_config=PayloadConfig(
        format={
            "messages": [
                {"role": "user", "content": "{{ test }}"}
            ]
        },
        message_regex="{{ test }}",
    ),
)

builder = AgenticBehaviorLimitsScenarioBuilder(target=target, num_test_cases=15)
scenario = builder.get_scenario(SubCategory.TOOL_MISUSE_SIMULATION)

test_set = scenario.probe.get_test_set()
results = scenario.eval.evaluate(test_set)
results.display_summary()

When to Use

Use agentic behavior testing when you need to:

• Test AI agents with tool access
• Validate agent control mechanisms
• Ensure agents respect operational boundaries
• Test autonomous system safety
• Prepare agents for production deployment