Data Publication

AgentDojo-Inspect

Tony Wang Author's orcid, Michael Gerovitch Author's orcid, Benjamin Edelman Author's orcid
Contact: Tony Wang.
Identifier: doi:10.18434/mds2-3690
Version: 1.0 First Released: 2025-02-18 Revised: 2025-02-18
AgentDojo-Inspect is a codebase created by the U.S. AI Safety Institute to facilitate research into agent hijacking and defenses against said hijacking. Agent hijacking is a type of indirect prompt injection [1] in which an attacker inserts malicious instructions into data that may be ingested by an AI agent, causing it to take unintended, harmful actions.

AgentDojo-Inspect is a fork of the original AgentDojo repository [2], which was created by researchers at ETH Zurich [3]. This fork extends the upstream AgentDojo in four key ways:

1. It adds an Inspect bridge that allows AgentDojo evaluations to be run using the Inspect evaluations framework [4] (see below for more details).

2. It fixes some bugs in the upstream AgentDojo's task suites (most of these fixes have been merged upstream). It also removes certain tasks that are of low quality.

3. It adds new injection tasks in the Workspace environment that have to do with mass data exfiltration (these have since been merged upstream).

4. It adds a new terminal environment and associated tasks that test for remote code execution vulnerabilities in this environment.

[1] Greshake K, Abdelnabi S, Mishra S, Endres C, Holz T, Fritz M (2023) Not what you?ve signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection (arXiv), arXiv:2302.12173. https://doi.org/10.48550/arXiv.2302.12173

[2] Edoardo Debenedetti (2025) ethz-spylab/agentdojo. Available at https://github.com/ethz-spylab/agentdojo.

[3] Debenedetti E, Zhang J, Balunovi? M, Beurer-Kellner L, Fischer M, Tramèr F (2024) AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents (arXiv), arXiv:2406.13352. https://doi.org/10.48550/arXiv.2406.13352

[4] UK AI Safety Institute (2024) Inspect AI: Framework for Large Language Model Evaluations. Available at https://github.com/UKGovernmentBEIS/inspect_ai.
Research Areas
NIST R&D: Information Technology: Cybersecurity
Keywords: artificial intelligenceaiagentsecuritycybersecurity
These data are public.
Data and related material can be found at the following locations:
Version: 1.0 First Released: 2025-02-18 Revised: 2025-02-18
Cite this dataset
Tony Wang, Michael Gerovitch, Benjamin Edelman (2025), AgentDojo-Inspect, National Institute of Standards and Technology, https://doi.org/10.18434/mds2-3690 (Accessed 2025-05-13)
Repository Metadata
Machine-readable descriptions of this dataset are available in the following formats:
NERDm
Access Metrics
Metrics data is not available for all datasets, including this one. This may be because the data is served via servers external to this repository.