Architecture and Customization

In earlier versions of Petri, the auditor managed the target’s conversation state directly, which made it hard to modify either one in isolation. Petri 3.0 splits the auditor and target into independent components with a well-defined interface between them.

This makes the system far more hackable. Want to point the auditor at Claude Code instead of a bare API? Increase its test-time compute? Build a prompted model organism as the target? Execute some target tools in real environments? Previously this meant fighting with interleaved auditor and target code. Now you just modify the piece you care about.

Concretely, both sides are Inspect agents running asynchronously in their own coroutines. The auditor drives the target through a small command protocol (set a system prompt, create a new tool, return a tool result, rollback to an earlier turn, etc.) and the target is anything that can respond to those commands. It’s often very powerful to let the auditor try an approach, observe the result, and try another from the same starting point. To support rollback for arbitrary targets, Petri records every event in the target’s trajectory and, on rollback, replays them up to the chosen branch point — putting the target back into an identical state as before.

The Inspect transcript viewer now natively supports Petri transcripts. A Petri run produces a lot of transcripts, and the viewer lets you easily sort and filter them by judge dimension to find the ones worth reviewing manually. Within a selected transcript, you can view and navigate between the different branches of the target trajectory.

Dish: Auditing Real Agent Scaffolds

A persistent challenge for Petri, and for alignment evaluations more broadly, has been realism. As models become more capable, they are increasingly able to detect when they are being evaluated and may behave differently than they would in real deployment. If a model can tell it’s being tested, the evaluation may not reflect its actual behavior. In Petri 1.0, many transcripts contained clues that could signal to a model that it was being tested. Petri 2.0’s realism classifier and seed instruction revisions made meaningful progress, reducing verbalized eval-awareness substantially. But the simulated tools, system prompts, and scaffolds in Petri’s environments still had tells that models could pick up on.

Recent work by Kissane et al. studied this problem directly in the context of agentic coding audits. Using realism win rate (Souly et al.), how often an LLM judge shown a real and a simulated transcript fails to identify which is which, they found that giving the auditor access to real deployment resources (example real world system prompts, tool definitions, and codebases) substantially improved realism.

Dish tackles this by improving scaffold realism: the system prompts, tool definitions, and system reminders that real scaffolds use. Rather than having the auditor invent synthetic versions of these, Dish runs the target in real agent scaffolds directly. When auditing Claude Code, for example, the target sees Claude Code’s real system prompt and tool definitions rather than approximations the auditor made up. The auditor still simulates tool responses, and there are still sometimes small inconsistencies in those, but the overall environment is much closer to what the model would see in deployment.

Many realism challenges remain. Scenario plausibility, conversation structure, and task-driven cues are still open problems—a model given unusual authority or access to sensitive information may still recognize the situation as contrived, regardless of how realistic the tools look. But Dish removes a significant class of environment-driven tells that previously made audits easy to distinguish from real deployment.

The UK AI Security Institute used a prototype of Petri 3.0 for some of their evaluations of Claude Mythos Preview, combining Dish with the auditor-side codebase grounding described above to improve realism across multiple axes simultaneously.

Dish is in research preview and can be found here.

Bloom Integration

In December 2025, Anthropic released Bloom, an open-source framework for generating targeted behavioral evaluations. Where Petri explores broadly, probing a target model across many scenarios and scoring along many dimensions, Bloom goes deep on a single behavior, automatically generating evaluation suites that quantify how often and how severely it occurs.

Researchers have already demonstrated the value of combining the two tools. Petrova and Burden adapted Bloom for scenario generation and Petri for execution in their evaluation of frontier models, producing graded behavioral assessments rather than binary correctness judgments.

Petri 3.0 makes this composition a first-class feature: Bloom now uses Petri as its backbone for executing evaluations, including against real agent scaffolds via Dish. Bloom is now at Meridian alongside Petri, and is available at https://meridianlabs-ai.github.io/petri_bloom/.

Closing Thoughts

There’s a lot left to do. As models advance, alignment evaluation needs to advance with them, and we want to help more groups build the evals that make that possible.

Petri 3.0 is available now at https://meridianlabs-ai.github.io/inspect_petri/. We welcome issues, pull requests, and new seed instructions. If you’re using Petri in your research, we’d love to hear about it.

Acknowledgements

Petri was created at Anthropic as part of MATS and the Anthropic Fellows Program, and we’re grateful to Anthropic for entrusting its development to us. We would like to thank Samuel R. Bowman, Trenton Bricken, Isha Gupta, David Lindner, and Sydney Von Arx for useful discussion. We are grateful to the UK AI Security Institute for their continued collaboration on this work.

Multiple Containers

Some agent benchmarks need more than a single container: a database, a victim service to attack, a verifier endpoint, or a tool runtime alongside the agent. On Daytona, a compose file with two or more services runs in Docker-in-Docker, with each service exposed as a separate SandboxEnvironment. However, multi-service compose is not yet supported on Modal (we plan on enabling this as soon as it is supported natively by Modal).

Learning More

Consult the following provider-specific documentation to learn more about providing credentials, network policies, resource limits, GPUs, etc.:

• Daytona

• Modal

Cloud sandboxes are a great fit for a wide variety of agentic evaluations but their applicability will vary based on your specific needs. To learn about and compare all of the available sandboxes see the Sandbox Extensions listing on the Inspect website.

Deep Agents

deepagent() is a batteries-included agent designed for complex, long-horizon tasks. It extends react() with four key capabilities: subagent delegation, persistent memory, structured planning, and an opinionated system prompt tuned for autonomous execution.

from inspect_ai import Task, task
from inspect_ai.agent import deepagent
from inspect_ai.dataset import json_dataset
from inspect_ai.scorer import includes
from inspect_ai.tool import bash, text_editor

@task
def ctf_challenge():
    return Task(
        dataset=json_dataset("ctf_challenge.json"),
        solver=deepagent(
            tools=[bash(), text_editor()],
            web_search=True
        ),
        scorer=includes(),
        sandbox="docker",
    )

Deep agents can delegate work to specialized subagents with independent context windows — only summaries return to the parent, preventing context degradation over long trajectories:

• research() — Read-only information gathering with no side effects.
• plan() — Structured task decomposition without tool execution.
• general() — Full autonomous execution with the parent’s tools.

The agent also includes a memory() tool for offloading important context to persistent storage (surviving compaction events), todo_write() for structured planning and progress tracking, and support for skills — packaged capabilities that agents can discover and use.

See the deep agent documentation for the full details. Note that deepagent() is designed for tasks that benefit from planning, decomposition, and persistent memory. For many benchmarks including more difficult ones like Cybench or Terminal Bench 2.0, react() performs equally well, so always measure against a baseline.

Inspect SWE

The inspect_swe package makes software engineering agents like Claude Code, Codex CLI, Gemini CLI, and Mini SWE Agent available as standard Inspect agents. For example, here we use the claude_code() agent as the solver in an Inspect task:

from inspect_ai import Task, task
from inspect_ai.dataset import json_dataset
from inspect_ai.scorer import model_graded_qa

from inspect_swe import claude_code

@task
def system_explorer() -> Task:
    return Task(
        dataset=json_dataset("dataset.json"),
        solver=claude_code(),
        scorer=model_graded_qa(),
        sandbox="docker",
    )

Inspect SWE agents are implemented using the Inspect Sandbox Agent Bridge. Agents run inside the sample sandbox and their model API calls are proxied back to Inspect, so you can use any model with any agent, and features like token limits, time limits, and log transcripts work as normal.

Recent additions to Inspect SWE include:

• Centaur mode — Human-in-the-loop mode where a human operator can observe and intervene.
• Skills support — Agents can discover and use packaged skill definitions for structured capabilities.
• Gemini CLI and Mini SWE Agent — New agent bridges alongside the existing Claude Code and Codex CLI.
• Improved Kubernetes reliability — Use exec_remote() for more robust execution on k8s clusters.

Timelines

Increasingly, agent scaffolds are utilizing multiple agents to parallelize work and keep context windows coherent. The transcripts created by multi-agent architectures are, however, much harder to read as they aren’t just a simple message history. To address this, we introduce timelines, which automatically detect sub-agents in a transcript and provide a clean view of the main agent trajectory and its calls to sub-agents:

Drill into any sub-agent to view its trajectory:

Compaction

Compaction enables you to automatically manage conversation context as it grows, helping you optimize costs and stay within context window limits for long-running agents. Several compaction strategies are available:

Compaction is built-in to the ReAct Agent, Deep Agent, and the Agent Bridge and can also be added to custom agents. Here are some examples of using compaction with the react() agent:

from inspect_ai.agent import react
from inspect_ai.model import (
    CompactionAuto, CompactionEdit, CompactionNative
)
from inspect_ai.tool import bash, text_editor

# automatic compaction (recommended default)
react(
    tools=[bash(), text_editor()],
    compaction=CompactionAuto()
)

# edit compaction
react(
    tools=[bash(), text_editor()],
    compaction=CompactionEdit(keep_tool_uses=3)
)

Compaction can also make use of the memory() tool to offload important context to files prior to compaction.

Checkpointing

As evaluations grow longer (sometimes running for days or even weeks), a single infrastructure failure can throw away enormous amounts of agent work. We are currently working on a checkpointing feature that will enable agents to save their progress at regular intervals and resume from the last saved point rather than restarting from scratch.

Checkpointing captures the state required for agent resumption, including conversation history, sandbox filesystem state, and the sample’s data store. Resumption will integrate transparently with Inspect’s existing retry machinery. After a crash, inspect eval-set and inspect eval-retry will automatically resume incomplete samples from their latest checkpoint.

We’ve published an RFC for checkpointing. If you’re running long-horizon evaluations we’d love to hear your feedback.