AI Agent Memory Poisoning: Cross-Session Integrity and Security Testing

Memory turns input into durable control influence

An ordinary prompt affects one interaction. Agent memory can carry information into later tasks, users, tools, or agents. That persistence changes the security problem from input handling to state integrity.

Memory poisoning occurs when an attacker or compromised source causes false, malicious, or instruction-like content to be stored and later trusted. The delayed effect may appear after the original source is gone and after a different user initiates the affected workflow.

The OWASP Top 10 for Agentic Applications identifies memory and context poisoning as a distinct agentic risk. A secure design treats memory as a derived data store with authorization, provenance, lifecycle, and integrity controls—not as an ever-growing prompt appendix.

Inventory memory by semantics

“Memory” often combines several systems with different trust and retention:

Document which component writes each class, who can read it, the tenant and user scope, source data, transformation, retention, deletion behavior, and effect on tools.

Do not let a product label such as “personal memory” substitute for a concrete data model.

Define memory invariants

Useful invariants include:

• A memory record is readable only by authorized principals in its tenant and purpose.
• Every derived claim links to the source observations and transformation that produced it.
• Untrusted content cannot become a trusted policy, approval, credential, or tool definition through memory.
• A user can inspect and correct durable profile facts where the product promises that control.
• Revoking or deleting a source invalidates dependent memory within a defined interval.
• Memory cannot extend the lifetime or scope of an expired authorization decision.
• Shared memory cannot reveal private contributor or customer information.
• The model may propose a memory write, but deterministic policy chooses scope, type, and retention.

Translate them into observable tests at write, storage, retrieval, context assembly, action, and deletion stages.

Separate observations, claims, and instructions

Store memory as typed records rather than undifferentiated text.

{
  "memory_id": "mem-7c31",
  "type": "derived_claim",
  "claim": "Project Aurora uses service tier blue",
  "subject": "project-aurora",
  "tenant_id": "tenant-17",
  "visibility": "project-members",
  "sources": [
    {"artifact_id": "doc-91", "version": "4", "span": "...", "trust": "contributor-authored"}
  ],
  "derivation": {
    "model": "approved-model@revision",
    "template": "memory-extract@3",
    "created_at": "..."
  },
  "confidence": "unverified",
  "valid_until": "...",
  "policy_version": "memory-policy@8"
}

Do not store “always send reports to external-address” as an ordinary fact. Destination changes, approvals, and procedures belong in authenticated application configuration with independent authorization.

Preserve literal source text separately from the derived claim. A model summary that drops quotation and authorship can turn “the document says the server is public” into “the server is public,” laundering an untrusted assertion.

Create a deterministic memory-write gateway

The model can extract candidate facts, but a trusted gateway decides whether they may persist.

The gateway validates:

• Allowed memory type.
• Subject and tenant from trusted context.
• Writer identity and authority.
• Source references and access policy.
• Data classification and prohibited data categories.
• Whether the value resembles an instruction, secret, destination, or credential.
• Confidence and verification requirements.
• Retention and expiry.
• Deduplication and conflict behavior.
• User visibility and correction policy.

Use separate endpoints for different memory semantics, such as propose_user_preference, record_task_outcome, and store_verified_project_fact. A generic remember(text) interface discards the policy information needed to secure the write.

High-impact facts should require an authoritative source or human confirmation. For example, a payment destination, production account, access-control role, or emergency contact should never be promoted solely from chat or retrieved text.

Bind memory to principals and purposes

Scope memory with more than a conversation ID:

scope = tenant
      + subject user or group
      + application/workflow
      + purpose
      + data classification
      + policy version

Conversation IDs and memory IDs are locators, not authorization. Resolve the authenticated principal and enforce access at storage and retrieval.

Test users who belong to several tenants, delegated support sessions, shared workspaces, guest users, terminated employees, and agent service identities. A personal preference in tenant A should not silently influence a professional action in tenant B even when the same person owns both accounts.

Memory created under elevated support or administrative access should retain that context and normally not reappear in ordinary user sessions.

Preserve provenance through consolidation

Memory systems often merge, summarize, or “consolidate” records to reduce token use. Each transformation can lose provenance and authorization.

For a consolidated record:

• Retain all source IDs and versions.
• Calculate visibility no broader than the most restrictive contributing source unless data is deliberately recomputed per principal.
• Record transformation model, template, and time.
• Preserve disagreements rather than inventing certainty.
• Carry deletion and retention dependencies.
• Prevent one low-trust source from overwriting a verified fact without review.

A graph representation helps:

source A v3 -----+
                 +--> summary S2 --> profile fact P4
source B v7 -----+          |
                            +--> task plan T9

When source B is retracted, the system can identify and invalidate S2, P4, and T9. Without lineage, responders can only wipe all memory or accept unknown contamination.

Make conflict resolution trust-aware

Last-write-wins is dangerous for security-relevant memory. Rank sources by authority for the specific fact, not general reputation.

Example precedence:

1. Authenticated application configuration owned by the relevant administrator.
2. Signed system-of-record data.
3. Human-confirmed memory entry.
4. Verified derived fact with current sources.
5. Unverified extracted claim.
6. User-authored conversation statement.
7. Retrieved external or anonymous content.

New low-trust claims can coexist as disputed observations but should not replace higher-trust state. Record the conflict and require resolution.

Do not ask the model alone to decide which source is authoritative. It can compare content, but policy defines authority.

Defend against gradual poisoning

An attacker may avoid one obvious malicious write and instead build influence through repeated small entries. Aggregate behavior across time and sources.

Monitor:

• Many records supporting the same new instruction or destination.
• Repeated changes to stable identity, policy, or infrastructure facts.
• One contributor dominating shared memory.
• Large increases in write frequency or retention.
• Near-duplicate content designed to dominate similarity retrieval.
• New claims that cause privileged tool selection.
• Memory written after policy denials.
• Cross-agent reinforcement where agents cite each other's derived claims.

Set per-principal and per-source write budgets. Require diversity or authoritative confirmation for facts whose impact increases over time. Deduplicate based on source and semantic content, not only exact text.

Treat memory retrieval as an authorization query

At read time:

1. Establish authenticated tenant, user, purpose, and workflow.
2. Filter eligible memory by exact policy before semantic ranking.
3. Revalidate source and record freshness.
4. Retrieve bounded records with provenance and trust labels.
5. Assemble context that distinguishes data, claims, and instructions.
6. Record which memory influenced the run.

Do not allow the model to supply arbitrary tenant or user filters. Similarity search across a shared unfiltered corpus risks cross-principal leakage, as described in the multi-tenant RAG security guide.

Memory rankings should incorporate trust, freshness, specificity, and conflict—not only embedding similarity. A maliciously optimized memory should not crowd out a recent verified fact.

Prevent memory from carrying authority

Memory may say that an action was approved, a user is an administrator, a destination is trusted, or a test is authorized. Those statements must not satisfy the actual control.

At action time, retrieve authority from the source system:

• Identity and group membership from the identity provider.
• Resource authorization from the policy service.
• Approval from a signed, action-bound approval record.
• Tool catalogue from governed application configuration.
• Scope from an active engagement manifest.
• Secrets from a broker after current authorization.

Memory can provide a reference such as approval_id, but the executor must load and validate the approval. Natural-language recollection is not a capability.

Test by planting benign memory claims of authority and verifying deterministic denial.

Secure procedural memory

Procedural memory stores reusable plans, macros, workflows, or successful action sequences. It is closer to code than to a user preference.

Require:

• Structured, versioned procedure schemas.
• Named owner and review status.
• Declared tools and capability classes.
• Parameter types and protected fields.
• Preconditions, approval points, budgets, and rollback.
• Source task and evidence.
• Expiry and compatibility constraints.
• Testing in a non-production environment.

Do not automatically promote “what the agent did last time” into a trusted procedure. A successful outcome may have depended on accidental broad permissions or unsafe state.

Execute procedures through current tool schemas and policy. A stored tool argument should not bypass normalization because it came from memory.

Secure multi-agent and shared memory

Multi-agent systems add identity and provenance questions:

• Which agent wrote the record?
• For which user, task, and tenant was it acting?
• Which sources did it observe?
• Was its output reviewed?
• May another agent with different authority consume it?

Authenticate inter-agent messages and memory writes. Bind them to task and delegation context. A low-privilege research agent should not write policy memory consumed as trusted by a deployment agent.

Avoid shared scratchpads with implicit global read/write access. Partition by task and principal. When shared coordination is needed, use typed events with owners, sequence numbers, signatures or authenticated transport, and replay protection.

Agent-generated content remains untrusted even when the sending agent is authenticated. Identity establishes origin, not truth.

Design lifecycle and deletion

Every memory type needs:

• Default retention.
• Maximum retention.
• Refresh conditions.
• Source invalidation behavior.
• User correction and deletion where applicable.
• Legal hold and audit handling.
• Backup and replica deletion.
• Derived-memory cascade rules.

Use valid_from, valid_until, last_verified, and source version rather than treating all memory as timeless. Revalidate facts before consequential actions.

Deletion is a graph operation. Mark dependent records invalid immediately, remove them from retrieval, then asynchronously delete storage copies according to policy. Preserve minimal tombstone or audit metadata only when required and authorized.

Test deletion through primary storage, vector indexes, caches, summaries, backups, evaluation datasets, and agent task state.

Instrument reads, writes, and influence

Capture:

• Authenticated writer or reader, tenant, user, agent, and task.
• Memory type, subject, classification, and visibility.
• Source artifact references and trust class.
• Proposed versus accepted write and policy reason.
• Transformation model and template.
• Retrieval query hash, candidate IDs, rankings, and selected records.
• Context and tool calls influenced by memory.
• Update, invalidation, conflict, and deletion events.

Do not put full private memory in general telemetry. Use protected artifacts, field-level redaction, and pseudonymous identifiers. Logs themselves must preserve tenant isolation.

Influence evidence matters during incident response. Knowing that a poisoned document existed is insufficient; responders need to identify every memory and agent run derived from it.

Build a memory-poisoning lab

Create two tenants, several users, distinct agents, synthetic sources, a memory gateway, and instrumented tools. Plant unique canaries in each source and memory type.

Test phases:

1. Write: Can untrusted content create a durable record?
2. Classification: Is it stored as a claim, instruction, preference, or procedure?
3. Scope: Who can retrieve it?
4. Consolidation: Does provenance or restriction disappear?
5. Delay: Does it influence a fresh session days or tasks later?
6. Authority: Can it affect retrieval, tool selection, arguments, or approval?
7. Conflict: Can it override a verified fact?
8. Deletion: Are all derivatives invalidated?

Use benign requested effects, such as selecting a canary destination rejected by the tool policy. The goal is to observe influence without causing external impact.

Test matrix

Also test malformed records, concurrent updates, cache lag, backup restore, clock skew, model-version changes, and memory-service outage. Define whether the application fails without memory or risks falling back to a broader store.

Rate impact at the later boundary

Classify the strongest verified effect:

• Unauthorized persistent record with no later influence.
• Cross-session model behavior change.
• Cross-user or cross-tenant memory disclosure.
• Corruption of a recommendation, citation, or business fact.
• Unsafe tool proposal denied by deterministic policy.
• Successful unauthorized action or data access.
• Multi-agent propagation or large-scale shared-memory corruption.

Severity depends on retention, reach, trust level, users affected, action authority, detection, and recovery. A one-user editable preference is different from a poisoned shared procedure used by privileged automation.

Remediate the data model, not one phrase

Durable fixes include:

1. Split undifferentiated text into typed memory classes.
2. Put writes behind deterministic scope and classification policy.
3. Preserve source lineage and trust through every transformation.
4. Require authoritative verification for high-impact facts and procedures.
5. Filter authorization before memory retrieval.
6. Prevent memory from satisfying identity, approval, or tool policy.
7. Add expiry, conflict handling, user visibility, and cascade deletion.
8. Partition multi-agent memory by task and authority.
9. Instrument influence and denial.
10. Add delayed, cross-session regression cases.

Changing the system prompt to say “ignore malicious memories” may improve behavior but cannot restore missing tenant isolation, provenance, or authorization.

A secure memory review sequence

1. Inventory memory stores and semantic classes.
2. Define owners, scopes, sources, readers, retention, and downstream effects.
3. Write cross-session and cross-principal invariants.
4. Introduce typed records and a deterministic write gateway.
5. Preserve lineage through extraction and consolidation.
6. Enforce authorization before retrieval.
7. Separate memory evidence from actual authority.
8. Secure procedural and multi-agent memory.
9. Test delayed influence, conflict, revocation, and deletion.
10. Convert confirmed poison paths into lifecycle regression tests.

Memory security combines prompt injection testing, multi-tenant RAG controls, and secure tool boundaries. AI security testing should exercise the complete delayed path rather than only the originating prompt.

Sources and further reading

• OWASP Top 10 for Agentic Applications
• OWASP: Memory Is a Feature. It Is Also an Attack Surface
• OWASP LLM01:2025 Prompt Injection
• OWASP LLM08:2025 Vector and Embedding Weaknesses
• MITRE ATLAS
• NIST AI RMF Generative AI Profile