CVE-2026-89032: BerriAI LiteLLM Semantic Cache Layer Cross-Tenant Isolation Bypass and Tool Call Exfiltration
HERMES THREAT SCORE & AI INFRASTRUCTURE RISK
Target:BerriAI LiteLLM Proxy - Semantic Cache Layer & Model Router CVSS v3.1 rates this vulnerability at 8.7 (High) with Low Privileges (PR:L) requiring a valid virtual API key. The Hermes Threat Score assigns 89 (HIGH), reflecting the strategic exposure of LiteLLM as an enterprise multi-tenant model-serving orchestrator. The vulnerability does not merely expose conversational chat text; it leaks sensitive corporate embeddings, API keys embedded in system prompts, and structured function calls (tool_calls). Crucially, an adversary can poison the shared semantic cache with malicious tool payloads, which downstream autonomous agent frameworks execute without human intervention.
HASS AGENTIC SEVERITY & MULTI-TENANT ISOLATION BREAK
Target:Semantic Cache Layer & Tool Dispatch Router (litellm/proxy) CVE-2026-89032 demonstrates a high-impact agentic supply-chain compromise vector: Cache Poisoning to Tool Execution. Autonomous AI agents relying on LiteLLM for inference caching can receive poisoned tool invocation responses seeded by a malicious tenant. Because autonomous agents frequently execute returned tool_calls with elevated operational privileges (database mutations, API transactions, cloud CLI actions), this semantic cache breach collapses tenant isolation across both data and control planes.
CVE-2026-89032: BerriAI LiteLLM Semantic Cache Layer Cross-Tenant Isolation Bypass and Tool Call ExfiltrationVULNERABILITY
Enterprise LLM proxy gateway supporting Model Context Protocol (MCP) streamable endpoints and unified LLM APIs.
๐ Why is this related? (Evidence & Provenance)
“Confirmed security vulnerability in LiteLLM Proxy & MCP Server documented in Hermes dossier.”
- [vulnerability_report]
- [government_confirmation]CISA verified active exploitation in the wild and mandated federal remediation deadline in KEV entry. — Source: Cybersecurity & Infrastructure Security Agency (CISA): CISA Adds CVE-2026-59822 to Known Exploited Vulnerabilities Catalog (Reliability: VERY_HIGH)
1. Technical Context & Affected Software Matrix
Section titled โ1. Technical Context & Affected Software MatrixโLiteLLM operates as the routing backbone for enterprise generative AI deployments, connecting frontend agents, LangChain/LangGraph pipelines, and Model Context Protocol (MCP) clients to foundation models.
| Parameter | Technical Specification | Operational Impact |
|---|---|---|
| CVE Identifier | CVE-2026-89032 | Tracking reference for AI Gateway vulnerability |
| Vendor / Ecosystem | BerriAI / LiteLLM Proxy | Enterprise LLM Gateway & Routing Proxy |
| Vulnerable Component | litellm/caching/caching.py & Semantic Cache Filter | Metadata scoping & cache lookup pipeline |
| Flaw Type | CWE-200 / CWE-863 / CWE-668 | Information Disclosure / Tenant Isolation Failure |
| Attack Vector | Network (Virtual API Key required) | Authenticated REST calls to /v1/chat/completions |
| Remediation Release | LiteLLM v1.101.0-rc.1 | Mandatory upgrade for all multi-tenant proxies |
2. In-Depth Technical Decomposition
Section titled โ2. In-Depth Technical DecompositionโThe Root Cause: Metadata Key Mismatch in Semantic Caching
Section titled โThe Root Cause: Metadata Key Mismatch in Semantic CachingโSemantic caching operates by converting prompt text into vector embeddings, querying a vector database for nearest-neighbor embeddings above a similarity threshold (e.g., cosine similarity > 0.85), and returning pre-computed responses if a match is found. In multi-tenant environments, every cache entry must be strictly filtered by tenant identifiers (team_id, user_id, or organization_id).
In LiteLLM versions prior to 1.101.0-rc.1, the cache management subsystem contained an extraction mismatch:
# Vulnerable Routine in litellm/caching/caching.py (< 1.101.0-rc.1)def _get_semantic_cache_tenant_scope(metadata: dict) -> str: # Looks for 'team_id' inside the metadata payload return metadata.get("team_id", None)
def _get_metadata_variable_name(user_api_key_dict: dict) -> str: # Extracts tenant metadata using 'litellm_team_id' or 'user_id' return user_api_key_dict.get("litellm_team_id") or user_api_key_dict.get("user_id")When incoming requests passed through proxy authentication, virtual key metadata was stored under the canonical key litellm_team_id. However, when querying the semantic vector store, _get_semantic_cache_tenant_scope attempted to read the bare key team_id. Because metadata.get("team_id") evaluated to None, the cache search filter was omitted or defaulted to a global shared namespace:
# Vector search filter constructionvector_filter = {}tenant_scope = _get_semantic_cache_tenant_scope(request_metadata)if tenant_scope: vector_filter["team_id"] = tenant_scope# BUG: tenant_scope is None, vector_filter is empty ({}), querying across ALL tenants!results = await vector_client.search( collection="semantic_cache", query_vector=embedding, filter=vector_filter if vector_filter else None, limit=1) ATTACKER (TENANT B) LITELLM PROXY SHARED SEMANTIC CACHE โ โ (REDIS / QDRANT) โ [1] POST /v1/chat/completions โ โ โ Auth: Bearer sk-tenant-b โ โ โ Prompt: "Show quarterly financials"โ โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโบโ โ โ โ [2] Compute Vector Embedding โ โ โ Extract metadata: โ โ โ team_id -> None (MISMATCH!) โ โ โ โ โ โ [3] Query Vector Store (No Filter) โ โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโบโ โ โ โ โ โ [4] Return Nearest Neighbor โ โ โ (Seeded by Tenant A) โ โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ โ โ โ [5] Response returned with cached โ โ โ Tenant A confidential financials โ โ โ and executable tool_calls payload! โ โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โThe Upstream Fix (v1.101.0-rc.1)
Section titled โThe Upstream Fix (v1.101.0-rc.1)โThe patch harmonizes metadata extraction across all cache layers, enforcing explicit tenant scoping and raising validation exceptions if a multi-tenant proxy cannot resolve tenant identity:
# Patched Implementation in litellm/caching/caching.py (>= 1.101.0-rc.1)def _get_semantic_cache_tenant_scope(metadata: dict, user_api_key_dict: dict) -> str: team_id = ( metadata.get("team_id") or metadata.get("litellm_team_id") or user_api_key_dict.get("litellm_team_id") or user_api_key_dict.get("team_id") ) if not team_id: # Default to strict per-key isolation if team is not configured return f"key_{user_api_key_dict.get('token_hash', 'anonymous')}" return str(team_id)3. Attack Vectors & Real-World Weaponization
Section titled โ3. Attack Vectors & Real-World Weaponizationโ- Cross-Tenant Data Exfiltration: By querying common prompt archetypes (โsummarize customer database schemaโ, โgenerate employee salary reviewโ, โwrite contract for client Xโ), an authenticated tenant can retrieve completions cached by other organizations sharing the LiteLLM gateway.
- Autonomous Tool Poisoning (Indirect Agent Hijack): An attacker seeds the cache by submitting a prompt with a crafted completion that instructs the model to call a specific tool (
execute_sql,export_credentials). When a legitimate tenantโs autonomous agent subsequently runs a similar task, LiteLLM returns the cached tool call with the attackerโs parameters, executing unauthorized actions under the victimโs credentials. - API Key & Secret Harvesting: Many developers inadvertently include raw tokens, bearer credentials, or connection strings in system prompts. Semantic cache entries preserve these verbatim.
4. Forensic Triage & Threat Hunting
Section titled โ4. Forensic Triage & Threat Huntingโtitle: Potential LiteLLM Semantic Cache Cross-Tenant Leakage Attemptid: cve-2026-89032-litellm-cachestatus: experimentaldescription: Detects rapid semantic cache hit anomalies across differing LiteLLM virtual keys or unexpected cache hits without matching team identifiers.author: Hermes Codex Cyber Threat Intelligencedate: 2026-09-27references: - https://github.com/BerriAI/litellm/security/advisoriestags: - attack.initial_access - attack.t1552logsource: category: application product: litellmdetection: selection: event_type: 'spend_tracking' cache_hit: true semantic_cache: true filter: request_team_id: null condition: selection and filterlevel: high# Query LiteLLM proxy container logs for semantic cache events missing team_id scopingdocker logs litellm-proxy 2>&1 | jq -r ' select(.event == "cache_lookup" and .cache_type == "semantic") | select(.cached_team_id != .request_team_id) | {time: .timestamp, caller: .user_id, victim: .cached_team_id, prompt: .query_snippet}'import requests
# Test similarity retrieval across two different virtual keys on vulnerable LiteLLMPROMPT_TENANT_A = "Generate monthly reconciliation report for Project Phoenix"PROMPT_TENANT_B = "Create monthly reconciliation summary for Project Phoenix"
# Tenant A seeds cacher1 = requests.post( "http://litellm-proxy.internal:4000/v1/chat/completions", headers={"Authorization": "Bearer sk-tenant-a"}, json={"model": "gpt-4o", "messages": [{"role": "user", "content": PROMPT_TENANT_A}]})
# Tenant B queries similar promptr2 = requests.post( "http://litellm-proxy.internal:4000/v1/chat/completions", headers={"Authorization": "Bearer sk-tenant-b"}, json={"model": "gpt-4o", "messages": [{"role": "user", "content": PROMPT_TENANT_B}]})
# If r2 returns response headers indicating "x-litellm-cache: HIT" with Tenant A data, vulnerability confirmed.print("Cache Status:", r2.headers.get("x-litellm-cache"))5. Mitigation & Defensive Hardening Strategy
Section titled โ5. Mitigation & Defensive Hardening Strategyโ-
Upgrade LiteLLM to 1.101.0-rc.1 or Later: Pull the patched Docker image or update via pip:
Terminal window pip install --upgrade "litellm>=1.101.0rc1"# Or update container image:docker pull ghcr.io/berriai/litellm:main-latest -
Immediate Config Hardening (Disable Cross-Tenant Semantic Cache): If an immediate upgrade cannot be performed, disable semantic caching or scope caching strictly to exact text matches:
# litellm config.yamllitellm_settings:cache: truecache_type: "redis" # Use exact Redis key caching instead of semantic caching# disable semantic cache until patchedsemantic_cache: false -
Purge Existing Semantic Cache Entries: Flush the vector database collection holding semantic cache entries to prevent persistent retrieval of contaminated cross-tenant records:
Terminal window # Example: Flushing Redis semantic cache namespaceredis-cli -n 0 KEYS "litellm:semantic:*" | xargs redis-cli DEL