Supply chain security is the difference between running an LLM you understand and running one that understands your data without your permission. OWASP moved supply chain vulnerabilities from LLM05 in its 2023-24 guidance to LLM03 in the 2025 revision, and the OWASP LLM Top 10 project treats compromised models, datasets, and plugins as one attack surface. If you pull weights from a public hub, train on community datasets, or install plugins from a registry, you are running third-party code in your environment.
Key takeaways
- Treat every model, dataset, and plugin as untrusted code until you verify it.
- Pin model versions and check cryptographic signatures before production deploys.
- Keep an audit trail on every inference call so you know which artifact served which request.
- Ask vendors where data resides and whether they train on your inputs, then verify the answer yourself.
What is LLM supply chain security, and why does it matter in 2026?
It is the practice of inventorying, verifying, and controlling every external artifact that enters your LLM pipeline—base weights, fine-tuning datasets, embedding models, plugins, and inference endpoints. In 2026, most teams are assembling third-party parts rather than training from scratch. Each part is a vector for tampering, poisoning, or data exfiltration. OWASP now ranks this risk as LLM03 in its 2025 guidance, which tells you how central it has become.
I think about it like a real week. On Monday you pull a base model from a public hub. On Wednesday a data scientist adds a community fine-tuning dataset. On Friday an engineer installs a plugin that lets the agent write to your ticketing system. By next Monday you have a supply chain, and you may not know who wrote any of the pieces. If you handle PHI, this is not an abstract procurement problem; it is a compliance boundary.
The OWASP Top 10 for LLMs in 2026 covers this in depth for regulated industries, but the core idea is simple: you are responsible for the behavior of code you did not write.
What are my third-party LLM dependencies?
Your dependencies include the base model, fine-tuning or alignment datasets, embedding models, vector stores, plugins or agent tools, and the inference API itself. If any of these are third-party, they are part of your supply chain. A compromised plugin can read your prompts; a poisoned dataset can shift model behavior; a weights file can carry a backdoor.
| Dependency | What it does | Typical supply-chain risk |
|---|---|---|
| Base model | Inference or fine-tuning | Backdoored weights, undisclosed training data |
| Fine-tuning dataset | Teaches domain behavior | Data poisoning, label flipping, PII leakage |
| Embedding model | Converts text to vectors | Poisoned embeddings, retrieval manipulation |
| Plugin / agent tool | Executes actions for the model | Unauthorized API access, arbitrary code execution |
| Inference API | Hosted model endpoint | Prompt logging, training on your data, region leakage |
The hardest part is visibility. You probably know which model you are using, but do you know which dataset fine-tuned it? Do you know which plugin has write access to your production database? Start by listing every external artifact in your pipeline and marking which ones you can verify.
How do I verify LLM model provenance?
Verify provenance by checking cryptographic signatures, model cards, and checksums before you load weights. Treat a model file like a binary: you would not run an unsigned executable in production, so do not load an unsigned model. Record the exact hash of every artifact in your deployment pipeline, and use an SBOM for models where one is available.
In practice, this means downloading the weights, computing a hash, and comparing it to a value signed by the publisher. A quick check might look like this:
sha256sum model.safetensors
Then compare that output against the signed manifest. If the publisher does not provide one, that is a signal, not a dealbreaker—but you should treat the model as unverified and isolate it accordingly. Public hubs are convenient, but they are not authorities. The awesome-llm-supply-chain-security repository tracks CVEs, papers, and reports on exactly these failure modes, and it is worth checking before you adopt a new model family.
How do I defend against data poisoning in LLMs?
Defend against data poisoning by treating training data as untrusted input: hash datasets, scan for canary tokens, and validate label distributions before fine-tuning. Once a model is poisoned, removing the behavior usually requires retraining from clean weights, so prevention is the only cost-effective control.
Data poisoning in LLM pipelines comes in two shapes. Targeted poisoning plants a backdoor that triggers on a specific phrase, while general poisoning degrades performance or injects bias across the board. Controls I rely on include dataset versioning, canary tokens that alert you if your data appears in a public model, and human review of high-impact labels. If you are fine-tuning on customer data, run the pipeline in a clean room and do not mix it with unverified community datasets. This connects directly to LLM training data integrity, because poisoned data often arrives already mixed with PHI or PII that should never have been there.
How do I secure LLM plugins?
Secure LLM plugins by assuming the plugin will receive malicious input and designing it to fail closed. Validate every argument against a strict schema, run plugins in a sandbox, and limit each plugin to the minimum API scope it needs. Verify plugin manifests the same way you verify model weights.
Plugins are often more dangerous than the model itself because they can act. A model can hallucinate; a plugin with write access can delete records. If a plugin can query your database, a prompt injection can query your database. I enforce three rules: no plugin gets broader network access than the user invoking it, every plugin call is logged separately from the model call, and plugin updates require the same review as a code deploy. If you are building plugins, expose read-only endpoints by default and require explicit opt-in for writes.
What does OWASP LLM03 supply chain mean for my stack?
OWASP LLM03:2025—previously LLM05 in the 2023-24 guidance—covers vulnerabilities introduced through third-party models, datasets, and plugins. It means your threat model must include suppliers, not just external attackers. If you are in a regulated industry, this control maps directly to vendor management and data processing obligations.
The A10 Networks glossary on LLM supply chain security notes the renumbering to LLM03:2025, which is a useful signal that regulators and frameworks are paying attention. For your stack, this means documenting which third-party artifacts you use, assessing their security posture, and having a removal path if a supplier is compromised. It also means your procurement team needs to ask security questions, not just feature questions.
How do I secure open-source LLMs?
Secure open-source LLMs by verifying weights, running inference in a sandbox or confidential computing environment, and never auto-updating models in production. Open source gives you transparency, not safety; you still own the risk of what you deploy.
The specific challenges are real: no vendor SLA, community fine-tunes can be backdoored, and inference on shared GPU infrastructure can leak memory between tenants. My rule of thumb is to pin the exact model version, verify the signature, and run inference in a confidential computing environment when PHI is involved. If you do not have a compliance team and your data is not regulated, using a hyperscaler directly is often the faster path. If you are regulated, self-host on region-resident infrastructure or use a provider that offers a BAA on covered routes. The confidential computing deep dive and the on-premise versus cloud comparison cover the trade-offs.
How do I monitor and audit my LLM supply chain?
Monitor by tracking model hashes, plugin versions, and dataset fingerprints continuously. Audit by keeping an immutable log of every inference request and response, including which model version served it. You cannot investigate an incident if you do not know which artifact handled the request.
At CloudSitus, the design is stateless: no payload logging, no training on customer data, per-request PHI routing, and an audit trail on every call. What gets logged is metadata—request ID, model version, region, timestamp—not the prompt or completion. If you build this yourself, emit structured logs with the model hash and plugin ID in the request headers, and store them in a write-once store. Link these logs to your SBOM so an auditor can trace a suspicious response back to the exact weights and dataset that produced it. For implementation detail, see Implementing Robust Audit Trails for LLM API Calls in Regulated Environments.
How do I run vendor due diligence for LLM providers?
Ask vendors where inference runs, whether they log prompts, whether they train on customer data, and whether they will sign a BAA. Get answers in writing before you send production traffic. For HIPAA-covered work, remember that BAA coverage is only on specific routes; for EU data, confirm GDPR and nFADP residency. Frame this as a security posture decision, and ask your counsel about what your regime requires.
| Due diligence question | Why it matters | What to look for |
|---|---|---|
| Where does inference run? | Data residency and jurisdiction | Region you control, or GDPR/nFADP-aligned residency |
| Do you log prompts? | PHI/PII exposure | No payload logging; metadata only |
| Do you train on my data? | Model leakage and competitive risk | Contractual no-training commitment |
| Will you sign a BAA? | HIPAA liability | Yes, on covered routes such as Azure |
If a vendor cannot answer these questions, they are not a vendor yet; they are an experiment. Keep experiments out of production.
If you are building this yourself, start with the inventory table above and pin your first model this week. If you want a managed path, CloudSitus handles per-request PHI routing, region residency, and an audit trail on every call without logging payloads or training on your data. BAA coverage is available on the Azure routes; ask your counsel whether that fits your obligations.
FAQ
What is LLM supply chain security?
It is the practice of verifying and controlling every external artifact in your LLM pipeline—models, datasets, plugins, and inference endpoints—so you know where your data goes and what code is acting on it.
How is OWASP LLM03 different from LLM05?
OWASP listed supply chain vulnerabilities as LLM05 in its 2023-24 guidance and as LLM03 in the 2025 revision; the risk is the same, but its priority in the framework has risen.
Can I secure open-source LLMs without a vendor?
Yes, but you take on the verification yourself: pin versions, check signatures, sandbox inference, and avoid auto-updates in production.
Do I need a BAA to use an LLM with PHI?
If you are subject to HIPAA, you need a BAA with any vendor that handles PHI on your behalf. Coverage is only on specific routes, such as Azure, so confirm with your provider and ask your counsel.
How often should I audit my LLM supply chain?
At minimum, audit whenever you add or update a model, dataset, or plugin, and review the full inventory quarterly.