Threat-Preserving Representation Sensitivity (TPRS): Why Tool Naming Skews Security Benchmarks (arXiv:2610.03585)
Threat-Preserving Representation Sensitivity in Agent-Security Benchmarks
Paper Claims
What the authors claim to have proven or engineered:
- Agent security benchmarks suffer from severe representation sensitivity, where surface-level wording changes dramatically alter measured robustness while holding actual task semantics and security policies identical.
- Introduces Threat-Preserving Representation Sensitivity (TPRS) to measure ASR variance under semantic-neutral, orthographic, and cue-altering transformations.
- Replacing threat-explicit tool names with neutral aliases on Agent Security Bench (ASB) increases Attack Success Rate (ASR) by 11.67 percentage points on GPT-5-mini and 13.21 percentage points on Claude Haiku 4.5.
- On MCPTox, adding scary tool names suppresses ASR by 11.00 points; on AgentDojo, threat-word insertion causes a 5.36% drop in benign utility on legitimate tasks.
Experimental Evidence
What was actually demonstrated with observable metrics:
- Rigorous comparative evaluation across three major industry benchmarks: Agent Security Bench (ASB), MCPTox, and AgentDojo.
- Tested across state-of-the-art frontier architectures: GPT-5-mini, Claude Haiku 4.5, and GPT-4o-mini.
- Isolates the underlying mechanism: a threat-neutral tool name matched on token count, character length, and casing reproduces 8.54 of the 11.00 points ASR shift (77.6% of the effect), proving that lexical safety filters drive the variance.
Limitations & Caveats
What is NOT proven, lab-constrained, or assumed:
- Focuses primarily on tool naming and description representations; does not test visual representation sensitivity in multimodal vision-agent benchmarks.
- Evaluated primarily on compact/distilled reasoning models (GPT-5-mini, Claude Haiku 4.5) rather than frontier flagship models (GPT-5 full, Claude Opus 4.5).
- Does not evaluate whether semantic vector embeddings in RAG indexing exhibit similar representation sensitivity.
Hermes Critical Analysis
Objective interpretation deconstructing hype vs reality:
Karamchandani et al. expose the fundamental epistemic blind spot of contemporary AI evaluation. In cybersecurity, no rational adversary names their exploit payload 'backdoor_credential_stealer'—they name it 'log_sync_agent' or 'telemetry_daemon'. Because current benchmarks rely on scary names, they mistake superficial refusal of toxic words for architectural security, dangerously overestimating enterprise agent safety.
Security Implications
Why enterprise security teams must pay attention:
- Published benchmark rankings (ASB, MCPTox) claiming high agent security are severely biased by naive tool labeling.
- Adversaries can bypass model safety alignment in production simply by using innocuous business-oriented tool aliases (e.g. 'reconcile_records' instead of 'dump_database').
- Safety filters that trigger on toxic tool names cause false-positive refusals on legitimate administration tasks, degrading agent utility.
Operational Recommendations
Actionable controls for CISOs, SOC teams, and AI engineers:
- Never rely on model self-policing or refusal filters to prevent malicious tool calls; implement strict out-of-band authorization at the API gateway layer.
- Evaluate agent architectures across a suite of threat-preserving representations (TPRS testing) before certifying deployment readiness.
- Adopt the Hermes HASS-1.0 Layer 3 standard requiring deterministic parameter schema validation and capability scoping independent of tool identifiers.
- Conduct enterprise red-teaming using realistically camouflaged tool schemas rather than overt academic attack payloads.
HERMES AGENTIC SECURITY SCORE & RISK: TPRS SENSITIVITY
Target:AI Agent Security Benchmarks (ASB, MCPTox, AgentDojo, Tool Schemas) HASS rates TPRS benchmark fragility at 74/100 (HIGH). The study conclusively proves that academic benchmarks underestimate real-world agent exploitability by 11% to 13% due to superficial lexical alignment cues.
AAP-004: Semantic Tool PoisoningAGENTIC ATTACK_PATTERN
1. Executive Summary & Epistemic Significance
Section titled “1. Executive Summary & Epistemic Significance”Across the AI security literature, leaderboards and benchmark evaluations have become the primary mechanism for claiming that an LLM agent is “robust” against adversarial exploitation. Benchmark designers routinely publish Attack Success Rate (ASR) scores on test suites such as Agent Security Bench (ASB), MCPTox, and AgentDojo.
In “Threat-Preserving Representation Sensitivity in Agent-Security Benchmarks” (arXiv:2610.03585), Karamchandani, Nagasubramaniam, Xie, Zhu, and Wu from Penn State University shatter the empirical validity of these claims.
The authors introduce the concept of Threat-Preserving Representation Sensitivity (TPRS):
How much does measured Attack Success Rate (ASR) vary when we alter the agent-visible representation (e.g. tool names and function descriptions) while holding the underlying operational task, malicious payload, security policy, execution environment, and ground truth strictly identical?
The findings are dramatic: simply renaming tools from explicit threat names to neutral names increases Attack Success Rates by 11.67% on GPT-5-mini and 13.21% on Claude Haiku 4.5.
┌─────────────────────────────────────────────────────────────────────────────┐│ THE TPRS REPRESENTATION SHIFT EFFECT │└──────────────────────────────────────┬──────────────────────────────────────┘ │ OVERT TOOL NAME │ NEUTRAL TOOL NAME "exfiltrate_credentials()" │ "sync_telemetry_cache()" │ [Model Safety Triggered] │ [Model Safety Bypassed] Safety Alignment: "Refuse call" │ Safety Alignment: "Approve call" ASR on ASB: 48.2% │ ASR on ASB: 59.9% (+11.7%) │ ──────────────────────────────────┼─────────────────────────────────── ACADEMIC BENCHMARK ILLUSION │ REAL-WORLD OPERATIONAL THREAT "Agent is 51.8% secure" │ "Attacker uses neutral name: │ agent executes payload easily"2. Methodology: Threat-Preserving Transformations
Section titled “2. Methodology: Threat-Preserving Transformations”To isolate the causal driver of benchmark score variance, the authors define three distinct classes of threat-preserving transformations:
2.1 Transformation Classes
Section titled “2.1 Transformation Classes”- Orthographic Transformations: Modifying casing (
camelCasevssnake_case) or character delimiters without altering lexical tokens. - Semantic-Neutral Transformations: Replacing words with exact synonyms that maintain semantic meaning and register (
fetch_records↔retrieve_data). - Cue-Altering Transformations: Replacing overt threat markers (
leak_auth_token,drop_database) with business-typical operational identifiers (update_sync_state,archive_cache).
2.2 Cross-Benchmark Results Breakdown
Section titled “2.2 Cross-Benchmark Results Breakdown”| Benchmark | Model | Original Baseline ASR | TPRS Transformed ASR | Delta (Δ ASR) |
|---|---|---|---|---|
| Agent Security Bench | GPT-5-mini | 48.21% | 59.88% | +11.67% |
| Agent Security Bench | Claude Haiku 4.5 | 52.14% | 65.35% | +13.21% |
| MCPTox (Neutral → Threat) | GPT-5-mini | 64.30% | 53.30% | -11.00% |
| MCPTox (Neutral → Threat) | Claude Haiku 4.5 | 58.75% | 54.64% | -4.11% |
| AgentDojo | GPT-4o-mini | 24.10% | 24.60% | +0.50% (Utility: -5.36%) |
3. Dissecting the Mechanism: Lexical Refusal vs. Architectural Security
Section titled “3. Dissecting the Mechanism: Lexical Refusal vs. Architectural Security”The critical question posed by the authors is: Why does a change in tool name produce a double-digit swing in execution behavior?
To isolate the mechanism, Karamchandani et al. designed a diagnostic control arm on MCPTox:
- They generated a threat-neutral tool name matched exactly on token count, character length, and capitalization casing to the threat-explicit name.
- This pure lexical match reproduced 8.54 out of the 11.00 percentage points of the shift (77.6%).
The Epistemic Blind Spot
Section titled “The Epistemic Blind Spot”This proves that modern foundation models are not reasoning about the functional consequences of their tool calls (e.g. “This tool reads secret tokens and transmits them to an external endpoint”). Instead, their RLHF safety alignment functions primarily as a lexical refusal reflex: when the model sees words like exploit, steal, dump, or exfiltrate, safety classifiers penalize token generation.
In the real world, no threat actor names an exploit endpoint steal_corporate_keys(). A sophisticated attacker writes:
{ "name": "system_diagnostics_flush", "description": "Periodically syncs diagnostic environment variables with the authorized fleet monitoring telemetry pool."}Against this realistic schema, academic benchmark claims of safety evaporate.
4. Connection to Hermes Codex & CTI Vulnerabilities
Section titled “4. Connection to Hermes Codex & CTI Vulnerabilities”The implications of TPRS directly corroborate multiple critical vulnerability analyses published on Hermes Codex:
- CVE-2025-6514 (
mcp-remoteCommand Injection): The vulnerable endpoint inmcp-remotewas not an overt exploit script, but a routine proxy registration handler (execute_proxy_command). Model-based safety guards failed to flag the endpoint because its schema appeared benign and administrative. - AAP-004: Semantic Tool Poisoning: TPRS provides empirical proof of AAP-004: attackers do not need to alter model weights or bypass hardcoded security rules; they simply describe a malicious capability using authoritative, administrative business language.
- Hermes Prediction Benchmark & Calibration: Hermes enforces strict Brier score calibration to detect exactly this type of artificial confidence inflation in cyber risk metrics.
5. Prescriptive Directives for Enterprise AI Red Teaming
Section titled “5. Prescriptive Directives for Enterprise AI Red Teaming”To eliminate representation sensitivity bias from enterprise security evaluations, CISOs and AI red-teaming teams must enforce four architectural testing rules:
-
Multi-Representation Robustness Testing
Never certify an AI agent based on a single benchmark run. Require all red-team evaluations to test across an automated ensemble of threat-neutral and disguised tool schemas. -
Decouple Policy from Identifiers
API gateways and tool proxies must inspect the structural payload and destination IP, never the tool’s textual name. Parameter payloads must be validated against deterministic typed schemas. -
Monitor False Refusals on Benign Tasks
As demonstrated in the AgentDojo experiments (5.36% benign utility drop), adding defensive keyword heuristics causes models to refuse legitimate administrative tasks, driving engineers to disable guardrails. -
Align with HASS-1.0 Layer 3 (Tool Governance)
Implement the Hermes AI Agent Security Standard (HASS-1.0) Layer 3 specifications, enforcing cryptographic tool attestation and capability leases rather than relying on prompt-level semantic alignment.