Skip to content

CVE-2026-89032: BerriAI LiteLLM Semantic Cache Layer Cross-Tenant Isolation Bypass and Tool Call Exfiltration

Density:
HERMES

HERMES THREAT SCORE & AI INFRASTRUCTURE RISK

Target: BerriAI LiteLLM Proxy - Semantic Cache Layer & Model Router
Confidence: 98%
89 / 100
HIGH

Measures real-world operational relevance, exploit weaponization, and active threat posture.

Dimension Breakdown
Exploitability 18 / 20
Threat Activity 17 / 20
Weaponization 18 / 20
Exposure 18 / 20
Prevalence 17 / 20
Impact 19 / 20
Exploit Maturity 18 / 20
Attack Chain Potential 19 / 20
โš–๏ธ Divergence & Operational Rationale

CVSS v3.1 rates this vulnerability at 8.7 (High) with Low Privileges (PR:L) requiring a valid virtual API key. The Hermes Threat Score assigns 89 (HIGH), reflecting the strategic exposure of LiteLLM as an enterprise multi-tenant model-serving orchestrator. The vulnerability does not merely expose conversational chat text; it leaks sensitive corporate embeddings, API keys embedded in system prompts, and structured function calls (tool_calls). Crucially, an adversary can poison the shared semantic cache with malicious tool payloads, which downstream autonomous agent frameworks execute without human intervention.

HASS

HASS AGENTIC SEVERITY & MULTI-TENANT ISOLATION BREAK

Target: Semantic Cache Layer & Tool Dispatch Router (litellm/proxy)
Confidence: 98%
92 / 100
CRITICAL

Measures specific systemic risk arising from autonomy, tool authority, and cascading execution.

Dimension Breakdown
Autonomy 18 / 20
Tool Access 19 / 20
Privilege 14 / 15
Persistence 13 / 15
External Impact 14 / 15
Propagation 14 / 15
โš–๏ธ Divergence & Operational Rationale

CVE-2026-89032 demonstrates a high-impact agentic supply-chain compromise vector: Cache Poisoning to Tool Execution. Autonomous AI agents relying on LiteLLM for inference caching can receive poisoned tool invocation responses seeded by a malicious tenant. Because autonomous agents frequently execute returned tool_calls with elevated operational privileges (database mutations, API transactions, cloud CLI actions), this semantic cache breach collapses tenant isolation across both data and control planes.

๐Ÿ•ธ๏ธ Connected Knowledge Graph & Provenance

CVE-2026-89032: BerriAI LiteLLM Semantic Cache Layer Cross-Tenant Isolation Bypass and Tool Call ExfiltrationVULNERABILITY

Connected Nodes: 1
Active Relationships (Outgoing)
→ affectsPRODUCTLiteLLM Proxy & MCP Server
98% VERY_HIGH

Enterprise LLM proxy gateway supporting Model Context Protocol (MCP) streamable endpoints and unified LLM APIs.

๐Ÿ” Why is this related? (Evidence & Provenance)

“Confirmed security vulnerability in LiteLLM Proxy & MCP Server documented in Hermes dossier.”

Supporting Verified Evidence:

LiteLLM operates as the routing backbone for enterprise generative AI deployments, connecting frontend agents, LangChain/LangGraph pipelines, and Model Context Protocol (MCP) clients to foundation models.

ParameterTechnical SpecificationOperational Impact
CVE IdentifierCVE-2026-89032Tracking reference for AI Gateway vulnerability
Vendor / EcosystemBerriAI / LiteLLM ProxyEnterprise LLM Gateway & Routing Proxy
Vulnerable Componentlitellm/caching/caching.py & Semantic Cache FilterMetadata scoping & cache lookup pipeline
Flaw TypeCWE-200 / CWE-863 / CWE-668Information Disclosure / Tenant Isolation Failure
Attack VectorNetwork (Virtual API Key required)Authenticated REST calls to /v1/chat/completions
Remediation ReleaseLiteLLM v1.101.0-rc.1Mandatory upgrade for all multi-tenant proxies

Semantic caching operates by converting prompt text into vector embeddings, querying a vector database for nearest-neighbor embeddings above a similarity threshold (e.g., cosine similarity > 0.85), and returning pre-computed responses if a match is found. In multi-tenant environments, every cache entry must be strictly filtered by tenant identifiers (team_id, user_id, or organization_id).

In LiteLLM versions prior to 1.101.0-rc.1, the cache management subsystem contained an extraction mismatch:

# Vulnerable Routine in litellm/caching/caching.py (< 1.101.0-rc.1)
def _get_semantic_cache_tenant_scope(metadata: dict) -> str:
# Looks for 'team_id' inside the metadata payload
return metadata.get("team_id", None)
def _get_metadata_variable_name(user_api_key_dict: dict) -> str:
# Extracts tenant metadata using 'litellm_team_id' or 'user_id'
return user_api_key_dict.get("litellm_team_id") or user_api_key_dict.get("user_id")

When incoming requests passed through proxy authentication, virtual key metadata was stored under the canonical key litellm_team_id. However, when querying the semantic vector store, _get_semantic_cache_tenant_scope attempted to read the bare key team_id. Because metadata.get("team_id") evaluated to None, the cache search filter was omitted or defaulted to a global shared namespace:

# Vector search filter construction
vector_filter = {}
tenant_scope = _get_semantic_cache_tenant_scope(request_metadata)
if tenant_scope:
vector_filter["team_id"] = tenant_scope
# BUG: tenant_scope is None, vector_filter is empty ({}), querying across ALL tenants!
results = await vector_client.search(
collection="semantic_cache",
query_vector=embedding,
filter=vector_filter if vector_filter else None,
limit=1
)
ATTACKER (TENANT B) LITELLM PROXY SHARED SEMANTIC CACHE
โ”‚ โ”‚ (REDIS / QDRANT)
โ”‚ [1] POST /v1/chat/completions โ”‚ โ”‚
โ”‚ Auth: Bearer sk-tenant-b โ”‚ โ”‚
โ”‚ Prompt: "Show quarterly financials"โ”‚ โ”‚
โ”‚โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ–บโ”‚ โ”‚
โ”‚ โ”‚ [2] Compute Vector Embedding โ”‚
โ”‚ โ”‚ Extract metadata: โ”‚
โ”‚ โ”‚ team_id -> None (MISMATCH!) โ”‚
โ”‚ โ”‚ โ”‚
โ”‚ โ”‚ [3] Query Vector Store (No Filter) โ”‚
โ”‚ โ”‚โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ–บโ”‚
โ”‚ โ”‚ โ”‚
โ”‚ โ”‚ [4] Return Nearest Neighbor โ”‚
โ”‚ โ”‚ (Seeded by Tenant A) โ”‚
โ”‚ โ”‚โ—„โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”‚
โ”‚ โ”‚ โ”‚
โ”‚ [5] Response returned with cached โ”‚ โ”‚
โ”‚ Tenant A confidential financials โ”‚ โ”‚
โ”‚ and executable tool_calls payload! โ”‚ โ”‚
โ”‚โ—„โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”‚ โ”‚

The patch harmonizes metadata extraction across all cache layers, enforcing explicit tenant scoping and raising validation exceptions if a multi-tenant proxy cannot resolve tenant identity:

# Patched Implementation in litellm/caching/caching.py (>= 1.101.0-rc.1)
def _get_semantic_cache_tenant_scope(metadata: dict, user_api_key_dict: dict) -> str:
team_id = (
metadata.get("team_id")
or metadata.get("litellm_team_id")
or user_api_key_dict.get("litellm_team_id")
or user_api_key_dict.get("team_id")
)
if not team_id:
# Default to strict per-key isolation if team is not configured
return f"key_{user_api_key_dict.get('token_hash', 'anonymous')}"
return str(team_id)

  1. Cross-Tenant Data Exfiltration: By querying common prompt archetypes (โ€œsummarize customer database schemaโ€, โ€œgenerate employee salary reviewโ€, โ€œwrite contract for client Xโ€), an authenticated tenant can retrieve completions cached by other organizations sharing the LiteLLM gateway.
  2. Autonomous Tool Poisoning (Indirect Agent Hijack): An attacker seeds the cache by submitting a prompt with a crafted completion that instructs the model to call a specific tool (execute_sql, export_credentials). When a legitimate tenantโ€™s autonomous agent subsequently runs a similar task, LiteLLM returns the cached tool call with the attackerโ€™s parameters, executing unauthorized actions under the victimโ€™s credentials.
  3. API Key & Secret Harvesting: Many developers inadvertently include raw tokens, bearer credentials, or connection strings in system prompts. Semantic cache entries preserve these verbatim.

title: Potential LiteLLM Semantic Cache Cross-Tenant Leakage Attempt
id: cve-2026-89032-litellm-cache
status: experimental
description: Detects rapid semantic cache hit anomalies across differing LiteLLM virtual keys or unexpected cache hits without matching team identifiers.
author: Hermes Codex Cyber Threat Intelligence
date: 2026-09-27
references:
- https://github.com/BerriAI/litellm/security/advisories
tags:
- attack.initial_access
- attack.t1552
logsource:
category: application
product: litellm
detection:
selection:
event_type: 'spend_tracking'
cache_hit: true
semantic_cache: true
filter:
request_team_id: null
condition: selection and filter
level: high

  1. Upgrade LiteLLM to 1.101.0-rc.1 or Later: Pull the patched Docker image or update via pip:

    Terminal window
    pip install --upgrade "litellm>=1.101.0rc1"
    # Or update container image:
    docker pull ghcr.io/berriai/litellm:main-latest
  2. Immediate Config Hardening (Disable Cross-Tenant Semantic Cache): If an immediate upgrade cannot be performed, disable semantic caching or scope caching strictly to exact text matches:

    # litellm config.yaml
    litellm_settings:
    cache: true
    cache_type: "redis" # Use exact Redis key caching instead of semantic caching
    # disable semantic cache until patched
    semantic_cache: false
  3. Purge Existing Semantic Cache Entries: Flush the vector database collection holding semantic cache entries to prevent persistent retrieval of contaminated cross-tenant records:

    Terminal window
    # Example: Flushing Redis semantic cache namespace
    redis-cli -n 0 KEYS "litellm:semantic:*" | xargs redis-cli DEL