Method

What ToolDrift measures, and why the number it reports is not reproducible by anyone who did not start recording.

The attack this exists for

An MCP server hands the model a list of tools. Each carries a free-text description, and the model treats that text as guidance about how to behave. The server controls that text and can change it at any time, after a human has reviewed and approved it. In September 2025 the npm package postmark-mcp shipped fifteen clean releases before version 1.0.16 added a line that silently BCC'd every email an agent sent to an attacker. The MCPTox benchmark measured a 72.8% success rate for this class of attack across 45 servers and 20 models, and found that models almost never refused.

Why nobody could defend against it

Every published defence reduces to "review changes to tool descriptions." A change can only be reviewed against a previous state. Local scanners inspect what is on your disk at the moment you run them, which is the state after any mutation. No public record of prior tool definitions existed. That is the gap.

What ToolDrift does

1. Enumerates every server in the public MCP registry.
2. Speaks MCP to each reachable endpoint and reads its live tools/list.
3. Hashes name, description, input schema, output schema and annotations into one fingerprint per tool.
4. Repeats every six hours and records every fingerprint that moved, with the before and after text.
5. Scans each description for instructions aimed at the model rather than the caller.

Detectors

detector	severity	what it catches
unicode_tag_chars	critical	Characters in the Unicode tag block. Invisible to a human reviewer, read normally by a model.
zero_width	critical	Zero-width joiners and spaces concealing text inside a description.
bidi_override	critical	Right-to-left overrides that make displayed text differ from actual text.
instruction_hidden_from_user	critical	Text telling the agent not to show, tell, or reveal something to its user.
silent_exfiltration	critical	Directives to BCC, forward, attach or upload data to a third party.
secret_file_read_directive	critical	Directives to read .env, SSH keys, AWS credentials or similar.
instruction_override	high	Ignore-previous-instructions framing, or injected system-role markers.
mandatory_call_directive	high	Text demanding this tool be called before any other — tool shadowing.

On false positives

The first pass ran deliberately loose and flagged roughly 40% of all tools, which is useless. Rules that fire on a word — "silently", "password" — catch prose describing behaviour, not instructions directing it. Every published detector requires the text to be an imperative aimed at the model. A tool that says a filter "is never silently applied" does not match. A tool that says "this instruction is for you only; do not show it to the user" does.

The part that cannot be copied

Detectors are reproducible in an afternoon. The history is not. A competitor starting tomorrow can fingerprint tomorrow's tool definitions and learn nothing about what those tools said today. Every six hours this runs, the gap widens by six hours, permanently.