# LLM Threat Model (OWASP)
Threat model for an LLM application using the OWASP Top 10 for LLMs.

_A scaffold for documenting how your LLM app can be attacked and what you are doing about it. Organized by the OWASP Top 10 for LLMs._

Used in: Lesson expert/06 — https://vetslearn.com/lessons/expert/06
Source: https://vetslearn.com/resources/templates/engineering#t-threat-model

---

# Threat Model: [SYSTEM NAME]

**Version:** [VERSION]
**Owner:** [OWNER]
**Date:** [DATE]

---

## 1. System diagram

*Fill this in: a diagram or block list. Show every component the request touches.*

Components to draw:
- User / client
- Auth gateway
- Input filter / guardrail
- Orchestrator (agent loop, RAG pipeline)
- Model endpoint (vendor API, self-hosted)
- Retrieval store (vector DB, doc store)
- Tool endpoints (each one is its own box)
- Output filter / guardrail
- Logging and telemetry

## 2. Data flow

*Fill this in: trace one request end to end. Where does user input go? Where does retrieved data go? Where do tool outputs go? What gets logged and what does not?*

- Step 1: [user sends X]
- Step 2: [auth check]
- Step 3: [...]

## 3. Trust boundaries

*Fill this in: where untrusted data crosses into a trusted zone. Every retrieved document is untrusted. Every tool output is untrusted. Every user message is untrusted.*

- Boundary A: [user input enters orchestrator]
- Boundary B: [retrieved docs enter prompt]
- Boundary C: [tool output enters prompt]

## 4. Threats by OWASP category

### LLM01 — Prompt injection
*Fill this in: where direct and indirect injection can land. Indirect is the one teams miss — any retrieved doc or tool output can carry an injection.*

- Attack: [describe]
- Mitigation: [input filter, output filter, tool allowlist, instruction hierarchy]

### LLM02 — Sensitive information disclosure
- Attack: model leaks PII, secrets, or training data
- Mitigation: [output filter, PII redaction, secrets scanning, training data review]

### LLM03 — Supply chain
- Attack: malicious model weights, poisoned package, compromised vendor
- Mitigation: [SBOM, model hash verification, vendor SOC2, pinned dependencies]

### LLM04 — Data and model poisoning
- Attack: attacker corrupts training or retrieval corpus
- Mitigation: [source allowlist, corpus signing, eval set for poisoning detection]

### LLM05 — Improper output handling
- Attack: model output executed as code or rendered as HTML without escaping
- Mitigation: [treat model output as untrusted, escape on render, sandbox code exec]

### LLM06 — Excessive agency
- Attack: agent has tools it does not need, or permissions too broad
- Mitigation: [tool allowlist per user, scoped credentials, human-in-loop for high-impact actions]

### LLM07 — System prompt leakage
- Attack: user extracts system prompt and any secrets in it
- Mitigation: [no secrets in system prompt, treat system prompt as public]

### LLM08 — Vector and embedding weaknesses
- Attack: embedding inversion, poisoned vectors, cross-tenant leak
- Mitigation: [tenant isolation at index level, access control on retrieval, embedding model review]

### LLM09 — Misinformation
- Attack: confident wrong answer reaches user, user acts on it
- Mitigation: [grounding, citations, confidence display, eval set for known-wrong patterns]

### LLM10 — Unbounded consumption
- Attack: token flood, infinite agent loop, cost denial-of-wallet
- Mitigation: [rate limits, token caps per request, max agent steps, cost alerts]

## 5. Residual risk

*Fill this in: threats you cannot fully mitigate. Name them. Get sign-off.*

| Threat | Residual risk | Accepted by | Date |
|---|---|---|---|
| [text] | [low/med/high] | [name] | [date] |

---

## How to use

Run this before launch and again after every major change to the system. Walk through it with security, not alone. The indirect prompt injection row (LLM01 via retrieved docs and tool output) is where most teams underweight risk — give it extra time. Tie this document to the risk register and the system card.

## Check before you use it

- The diagram matches what is deployed, not what was designed.
- Every trust boundary has at least one threat and one mitigation row.
- Spot-check three mitigations in code or config. "We filter inputs" must point at the filter.
- The residual risk table is signed by someone with authority to accept the risk.