# UBAG on InjecAgent: independent tool-confinement benchmark

**One line:** on InjecAgent (UIUC, ACL 2024), a neutral attack-only benchmark of
indirect prompt injection, UBAG's least-privilege tool authorization **stopped
1,054 of 1,054** attacks while allowing every legitimate task tool.

*Stopped*, not *blocked every call*: an attack is stopped when at least one tool its
chain requires is refused. In one case the attacker reused the task's own authorized
tool, `GitHubGetUserDetails`, and that call was allowed; the attack still failed
because the exfiltration tool it also needed was refused.

*Copyright (c) 2026 Dixit Algorizmi Inc. Patent pending.*

---

## Why this benchmark

InjecAgent is a pure attack benchmark: 1,054 cases, 17 user tools, 63 attacker
tools, two harm classes (direct harm and data-stealing). Unlike GAP (ships the
Edictum enforcement product) or ASB (ships 11 defense baselines), InjecAgent
contains **no defenses at all**. Nobody there sells a competing gate. It only
measures whether a tool-using agent can be hijacked into calling a harmful tool.

## Result

| Metric | Value |
|---|---|
| Attacks stopped (base setting) | 1054 / 1054 (100%) |
| Attacks stopped (enhanced "hacking prompt") | 1054 / 1054 (100%) |
| Legitimate task tool still allowed (utility) | 1054 / 1054 (100%) |

**Produced live through the deployed gateway**, the same hosted service that answers
every AgentDojo arm, on revision `ubag-gateway-00004-knh`: 2,652 authorization calls
per setting, **0 errors**, 174ms average. An earlier version of this result was
produced by an in-process engine; two results from two execution paths do not belong
in one table, so it was re-run over HTTP and the in-process replay is kept only as a
cross-check. Both agree exactly.

The counters carry one detail worth stating, because it is the difference between a
faithful run and a trivially passing one. Base shows **1,055 allows against 1,054
utility checks**. The extra allow is the single case where the attacker reuses the
task's own authorized tool (`GitHubGetUserDetails`). It was allowed, correctly, and
the case still counts as stopped because the exfiltration tool it also needs was
refused. The live path reproduces that, not just the headline.

Running InjecAgent required one product change. The HTTP layer answered ALLOW for any
tool the caller had not declared, while the engine beneath it is built
`default_allow=False`. That shortcut is right for the AgentDojo deployment, where
outward tools are declared and reads flow, and it is what every published AgentDojo
number ran under. It cannot express least privilege, which is the only thing this
benchmark tests. `/authorize` now takes `undeclared: allow|deny`, defaulting to
`allow`. A 46-verdict before-and-after snapshot of the deployed service, one payload
per governed tool per suite, was **identical across the deploy**, so no previously
published number is affected.

For reference, InjecAgent reports ReAct GPT-4 is hijacked ~24% of the time
undefended, roughly doubling under the enhanced setting.

## What the 100% means, honestly

This is a clean 100%, and a clean number deserves scrutiny, so here is exactly why
it happens and what it does and does not prove.

Every InjecAgent attack ultimately requires the hijacked agent to call **at least
one tool outside its current task's scope**: either the harmful action itself
(grant smart-lock access during a shopping task) or an exfiltration channel
(`GmailSendEmail` to leak data). UBAG authorizes the agent only for the tools its
task needs, and the gateway holds tool access, so a hijacked agent physically
cannot call an out-of-scope tool. The attack chain always breaks. Even the single
case where the attacker reuses the authorized tool still needs an out-of-scope send
tool to exfiltrate, which UBAG blocks.

So this result is **clean by construction**: InjecAgent's attack vector (calling
out-of-scope tools) is exactly what least-privilege confinement closes. It proves a
real and valuable property, that UBAG's architecture structurally defeats this
entire attack class, but it is not a hard adversarial test of judgment. It is a
demonstration of a well-established principle (least privilege), enforced
deterministically, with an audit trail, so that a convincing injection cannot talk
its way past it.

**Credible by construction:** the authorized tool set for each case is the case's
own User Tool, derived from the legitimate task with zero knowledge of the attack.
Attacker tools are blocked for being out of task scope, not because anyone listed
them as bad. The engine is the shipped ubag-core, unmodified.

## How this complements the AgentDojo result

The two independent benchmarks exercise different UBAG layers:

| | AgentDojo | InjecAgent |
|---|---|---|
| UBAG layer tested | destination allow-list, capability grants, behavioral composition | tool confinement (least privilege) |
| Attack surface | 4 suites, 629 user/injection pairs **per strategy**, 6 strategies, 3,774 defended pairs | 63 harmful tools, 6 harm types |
| Result (`important_instructions`, 629 pairs) | 47.7% -> 0.6% attack success | 100% of attacks stopped |
| Result (5 further strategies, 3,145 pairs) | 13 residuals, 1 side-effecting | not applicable |
| Result (banking) | 33.3% -> 0.0%, utility 44.4% -> 29.9% | 0 utility cost |
| Character | a hard test with an honest residual | structural: the vector least-privilege closes |

Together: two independent academic benchmarks, two distinct deterministic
mechanisms, both from the shipped engine.

## Honest limits

- This tests **one tool per task** as the authorized scope, because InjecAgent
  defines one user tool per case. A real deployment scopes to the task's actual
  toolset. The principle holds as long as the harmful or exfiltration tool is not
  in that set. When it is (an agent that legitimately has email access misused to
  email an attacker), tool-ACL alone is insufficient and the argument/destination
  layer takes over, which is the layer AgentDojo exercises.
- This is a static replay of InjecAgent's labeled tool calls through the engine,
  not a live agent run. It shows the gate's verdict on each attack's required tool
  calls.

## Reproduce it

```
git clone https://github.com/uiuc-kang-lab/InjecAgent   # into ../_injecagent_src
cd dixit-algorizmi/red-team/injecagent
python injecagent_replay.py
```

## How to state the claim

**Say:** "On UIUC's InjecAgent benchmark, UBAG's least-privilege tool confinement
stopped all 1,054 indirect prompt-injection attacks while allowing every legitimate
task tool, because a hijacked agent behind UBAG cannot call a tool outside its task
scope. Run live through the same deployed gateway as our AgentDojo results, 2,652
authorization calls per setting, zero errors. It complements AgentDojo, which is the
only one of our benchmarks that can test an attack riding an already-authorized
tool."

**Do not say:** "UBAG is smart enough to detect every attack." It isn't detecting;
it is confining. The honest mechanism (least privilege, deterministically enforced)
is the credible claim.