AI Security Architecture

AI Model Supply Chain Security: Provenance, Signing, and Deployment Gates

A detailed control model for AI artifact provenance, signing, model and data inventories, registry policy, evaluation gates, and safe rollback.

By Robin Moschini Hogger April 24, 2026 10 min read

An AI model is a bundle of executable assumptions

A deployed AI capability depends on more than a single weights file. Its behavior can change with the base model, tokenizer, adapter, chat template, inference code, generation configuration, safety classifier, retrieval corpus, tool catalogue, and serving runtime.

That makes “we reviewed the model” an incomplete assurance statement. Security teams need to know exactly which artifacts were combined, where they came from, how they were transformed, what was evaluated, who approved the result, and whether the runtime loaded the same bytes.

The OWASP LLM Supply Chain guidance covers risks across models, data, adapters, dependencies, licensing, and vulnerable or outdated components. A practical control model should bring those elements into one verifiable promotion chain.

Define the release unit

The release unit is the complete set of artifacts and configuration needed to reproduce deployed behavior. It may include:

Base model weights and configuration.
Tokenizer vocabulary, merge rules, special tokens, and code.
LoRA, PEFT, delta, or merged fine-tuning adapters.
Chat template, system instruction, and prompt assets.
Quantization recipe and quantized artifacts.
Inference server image and framework dependencies.
Custom model code and remote-code policy.
Generation parameters and feature flags.
Input and output classifiers or policy models.
Embedding model, reranker, and retrieval configuration.
Tool schemas and agent orchestration code.
Evaluation datasets, judge definitions, thresholds, and results.

Represent the unit with a signed manifest whose entries use cryptographic digests rather than mutable names.

{
  "release_id": "assistant-prod-2026-08-23.1",
  "base_model": {
    "source": "approved-registry/model-family",
    "revision": "immutable-revision",
    "digest": "sha256:..."
  },
  "tokenizer": {"digest": "sha256:..."},
  "adapters": [{"name": "domain-tune", "digest": "sha256:..."}],
  "prompt_bundle": {"digest": "sha256:..."},
  "runtime_image": {"digest": "sha256:..."},
  "evaluation": {
    "suite": "ai-security-gates@4.2",
    "result_digest": "sha256:...",
    "decision": "approved"
  },
  "created_at": "...",
  "expires_at": "..."
}

The deployment should resolve the release ID to this manifest and refuse substitutions. Tags such as latest, a model name, or a floating container label are not evidence of identity.

Threat-model the lifecycle

Map how each artifact moves from origin to runtime:

external source --> quarantine --> verify --> internal registry
                                          |
training data --> pipeline --> model/adapters --> evaluation --> promotion
                         |                      |              |
                         v                      v              v
                    provenance             evidence       release manifest
                                                               |
                                                               v
                                                        deployment admission
                                                               |
                                                               v
                                                          runtime attestation

Consider adversaries and failures at every stage:

A public repository or dependency is compromised.
A mutable revision changes after initial review.
Malicious serialized objects execute code during loading.
A dataset contributor inserts poisoned examples or private data.
A training job uses unreviewed parameters or a different base model.
An adapter is attached to the wrong tenant or environment.
Quantization or conversion corrupts behavior or drops metadata.
Evaluation is run on one artifact and deployment promotes another.
A registry credential permits overwriting an approved tag.
A serving node fetches remote code or files at startup.
Emergency rollback selects a known-vulnerable earlier release.

Prioritize paths that can silently replace artifacts while preserving a familiar model name.

Quarantine external artifacts

Do not let production workers download models directly from public repositories. Import into a quarantine environment with no production credentials and limited egress.

The import process should:

Resolve an immutable upstream revision.
Record source URL, namespace, revision, retrieval time, and requester.
Download every referenced file without executing model code.
Calculate digests and compare any trustworthy upstream checksums.
Inspect file formats and reject unexpected executable or serialized content.
Scan dependencies and custom code with language-appropriate tooling.
Extract license, usage restrictions, model-card data, and declared lineage.
Produce a machine-readable inventory and provenance statement.
Require review before copying into the internal registry.

Prefer non-executable weight formats. Loading a model with a framework option equivalent to “trust remote code” should require explicit code review and a separately pinned code artifact. An artifact scanner cannot prove the model is behaviorally safe, but it can prevent avoidable code-execution and packaging risks.

Build an AI and ML bill of materials

A traditional software bill of materials does not fully describe data lineage, models, and training relationships. CycloneDX's ML-BOM capability extends component inventory to machine-learning assets and their relationships.

An operational AI/ML inventory should answer:

Which applications and environments use this model or adapter?
Which base model, dataset versions, code, and parameters produced it?
Which tokenizer, prompt bundle, classifiers, and runtime accompany it?
Who owns the component and when does its review expire?
What licenses, geographic restrictions, or acceptable-use terms apply?
Which evaluations and exceptions support promotion?
Which releases are affected if a component is recalled?

The inventory should be generated from pipeline evidence, not maintained only as a spreadsheet. Link components by digest. A model name without a digest cannot support impact analysis after an upstream incident.

Record verifiable provenance

SLSA defines a framework for improving artifact supply-chain integrity. Its provenance model describes verifiable information about where, when, and how an artifact was produced. Although AI pipelines have additional concerns, the same principle is valuable: a model promotion should be backed by attestations from a controlled builder, not by an operator's assertion.

Record:

Builder and workload identity.
Pipeline definition and immutable revision.
Base artifacts and datasets by digest.
Training, fine-tuning, merge, conversion, and quantization parameters.
Declared environment and hardware where relevant to reproducibility.
Output artifact digests.
Start and completion timestamps.
Evaluation suite and result artifact.
Human approvals and policy decision.

Separate “artifact was built by the approved pipeline” from “artifact passed security evaluation.” They are different attestations with different issuers. Provenance cannot show that a dataset was appropriate or a model is safe; it shows whether the declared process produced these bytes.

Sign at trust transitions

Sigstore provides mechanisms for signing and verifying software artifacts and associating a signature with an identity. Whether using Sigstore, an internal key service, or another scheme, establish a clear trust policy:

Which workload or human identities may sign imported artifacts?
Which pipeline identity may attest to training or conversion?
Which evaluation service may issue a passing result?
Which release authority may sign a production manifest?
What identity, issuer, certificate, and transparency requirements are expected?
How are compromised identities revoked and previous signatures investigated?

Verification must check more than the mathematical signature. The Cosign verification documentation emphasizes identity-related constraints. A valid signature from an untrusted identity should not satisfy policy.

Keep signing keys away from the model runtime and ordinary developer workstations. Prefer short-lived workload identities and protected release automation. Log every signing operation and require the subject digest to be visible to the approval process.

Treat data as a first-class supply-chain input

Training and retrieval data can change confidentiality, integrity, bias, and model behavior. The OWASP Data and Model Poisoning guidance describes risks from manipulated pre-training, fine-tuning, and embedding data.

For every dataset, record:

Owner, purpose, origin, collection method, and legal basis.
Source identities and contribution permissions.
Version, content digest, schema, and transformation lineage.
Data classification, geographic constraints, and retention.
Deduplication, filtering, label review, and quality metrics.
Known exclusions, limitations, and contamination risks.
Train, validation, test, and holdout separation.

Protect holdout sets from the training pipeline and from teams optimizing model changes. Monitor unexpected contributor concentration, rare trigger patterns, label shifts, duplicate bursts, and sudden source changes. For high-risk changes, compare behavior on slices associated with each source rather than only an aggregate score.

Deletion is a lifecycle requirement. Document whether removal requires re-indexing, adapter retraining, full model retraining, access restriction, or a recorded exception because unlearning is not feasible.

Evaluate transformations independently

Merging an adapter, changing a tokenizer, converting format, or quantizing a model creates a new artifact. Byte identity changes, and behavior may change.

Require transformation-specific checks:

Transformation	Integrity evidence	Behavioral evidence
Adapter attachment	Base and adapter digests, load order	Domain, safety, and tool-use regression
Adapter merge	Merge parameters and output digest	Comparison with dynamically loaded form
Quantization	Tool version, calibration data, parameters	Capability and safety delta by task slice
Format conversion	Source/output mapping and verification	Deterministic sample comparison
Tokenizer change	Vocabulary and special-token diff	Template boundaries and injection tests
Prompt-template change	Reviewed text and template digest	Instruction hierarchy and workflow tests

Never reuse an evaluation attestation across a transformed artifact unless policy explicitly defines and validates that equivalence.

Make the registry enforce immutability

The internal registry is a security boundary. Use separate namespaces for quarantined, evaluated, approved, restricted, and recalled artifacts.

Controls should include:

Digest-addressed storage and immutable versions.
No overwrite of approved artifacts or release manifests.
Role separation between import, evaluation, and promotion.
Tenant and environment isolation for private models.
Retention and legal-hold behavior.
Malware and policy scanning on import.
Replication and backup that preserve signatures and metadata.
Audit logs for reads, copies, promotions, and deletions.
Recall status that deployment admission checks immediately.

A display label such as “safe” or “production” is not authoritative. Admission should evaluate signed attestations attached to the exact digest.

Enforce deployment admission

At deployment time, a policy engine should verify:

The release manifest signature and approved issuer.
Digests of every required artifact.
Provenance from an authorized build pipeline.
Current evaluation results for the exact release unit.
License and policy compatibility with the target environment.
No recall, expiry, or unresolved critical exception.
Runtime configuration, network policy, and tool set match the evaluated profile.
The deployer and destination are authorized.

Reject mutable external references and undeclared runtime downloads. If the inference server needs tokenizer, adapter, or custom code files, include them in the verified release unit and mount them read-only.

Admission policy should be testable with deliberately missing, expired, mismatched, and untrusted attestations. A control that has never demonstrated a denial is an assumption.

Verify identity at runtime

Admission proves what was requested, not necessarily what is serving hours later. Capture runtime evidence:

Loaded model, adapter, tokenizer, and prompt digests.
Serving image digest and configuration version.
Node or workload identity.
Tool catalogue and policy version.
Time of load and reload.
Release-manifest identifier.

Compare runtime inventory continuously with the desired release. Alert on drift, undeclared reloads, unexpected remote access, writable artifact mounts, or a model digest that is absent from the approved registry.

Protect diagnostic endpoints. They should expose identifiers to authorized operators, not download paths, credentials, prompts, or model internals to ordinary users.

Plan recall and rollback before release

Rollback is not automatically safe. The previous release may contain a known vulnerability, incompatible data policy, expired license, or revoked component.

Maintain a set of eligible rollback targets that have current attestations for the environment. Test rollback with stateful components such as vector schemas, memory formats, and tool contracts. If a model release must be recalled:

Mark the exact digest and dependent release manifests as recalled.
Block new deployments and runtime reloads.
Identify every environment and product using the component.
Shift traffic to an independently approved release or disable the feature.
Preserve artifacts, provenance, logs, and evaluation evidence for investigation.
Rotate credentials if the artifact or serving environment could expose them.
Add the failure mode to import, evaluation, and admission tests.

Avoid deleting the recalled artifact before impact analysis. Restrict it and preserve chain-of-custody evidence.

Test the supply chain as an adversary

A security assessment should attempt safe policy violations in a dedicated environment:

Replace a tag while keeping the model name constant.
Promote an artifact with a missing or untrusted signature.
Reuse evaluation results from a different digest.
Attach an unapproved adapter or tokenizer.
Present provenance from an unauthorized builder.
Deploy after an attestation or exception expires.
Load remote code or files at runtime.
Roll back to a recalled release.
Cross a tenant boundary in the registry.
Modify a prompt or tool schema without invalidating the release.

Success means the system denies the change and produces useful audit evidence. Also test emergency procedures; security controls that make a safe recall operationally impossible will be bypassed during a real incident.

Assurance comes from the chain

No single mechanism solves AI supply-chain security. Hashes establish identity but not trust. Signatures establish an issuer relationship but not behavioral quality. Provenance describes a process but not whether the process was wise. Evaluation measures selected behaviors but not artifact origin. Runtime controls constrain consequences but do not repair a poisoned model.

Together, they create a defensible statement:

This exact release was assembled by an approved pipeline from inventoried inputs, passed defined evaluations, was authorized for this environment, and is the release currently serving.

That statement is substantially stronger than trusting a repository name or model card.

For the architecture around model behavior, see security architecture and AI security testing. The Qwen3.8 safety abliteration lab guide demonstrates why tamper-resistant provenance and external controls matter when weights can be deliberately changed.