Concepts · Humans and machines
A layered guide to RAM, ROM, VRAM, bandwidth, context windows, KV caches, model weights, agent memory, and experience-shaped Emergent Memory.
Review pending. Report issues to editor@achiral.ai.
All Kinds of Memory in AI & Software
Human activity leaves records through time: a tool uncovered by an archaeologist, a mark on a wall, a photograph in a family album. Each can preserve evidence of an event. Retrieving the record gives us something to interpret; it does not guarantee a complete or objective reconstruction of the past.
Our own remembering helps us understand who we are, what we keep private, what we share, and what we do next. In Achiral's canonical definition, memory is a persistent change in the state of a system caused by prior experience that alters its future behavior, responses, or internal dynamics. The photograph is a record. The way remembering that afternoon changes a later choice is the behavioral question.
Software uses “memory” for several other things too: chips that hold bytes, a program's address space, a model's context, and knowledge encoded through training. This guide separates those meanings. It covers the main families and common neighboring terms; the Canonical Memory Glossary provides a wider alphabetical reference.
Start with the layer, not the word
These categories overlap, but they answer different questions:
| Layer | Question | Examples |
|---|---|---|
| Hardware | Where can bytes be held? | DRAM, SRAM, VRAM, flash. |
| Performance | How much fits, and how quickly can it move? | Capacity, bandwidth, latency. |
| Software | How does a running program address and manage memory? | Virtual memory, heap, stack, paging. |
| Model execution | What data occupies memory during computation? | Weights, activations, gradients, KV cache. |
| Inference context | What information is available for the current computation? | Context window, task state, working memory. |
| Retained knowledge | How is information preserved and made accessible? | Parameters, databases, episodes, retrieval. |
| Behavioral memory | What changes because of experience? | Salience, learned action patterns, Emergent Memory. |
A model's KV cache may occupy GPU memory implemented with HBM. Those are three descriptions of the same system at different layers, not three interchangeable kinds of remembering.
Hardware memory: RAM, DRAM, SRAM, and ROM
Random-access memory (RAM) permits access to addressed locations without reading everything before them. Dynamic RAM (DRAM) requires refresh; static RAM (SRAM) retains its state while powered without that refresh. Main system memory commonly uses DRAM, while processor caches commonly use SRAM. Samsung's DRAM overview distinguishes RAM and its technology families.
| Term | Meaning |
|---|---|
| SDRAM | Synchronous DRAM, coordinated with a clock. |
| DDR SDRAM | Double-data-rate SDRAM; DDR generations belong to this family. |
| LPDDR | Low-power DDR, used where power and space matter. |
| GDDR | Graphics double-data-rate memory. |
| HBM | High-bandwidth memory, using stacked DRAM and a wide interface. |
| eDRAM | DRAM embedded within a chip. |
| PSRAM | Pseudo-static RAM: DRAM with refresh handled internally. |
Read-only memory (ROM) names a different family. Some variants are fixed during manufacture; others can be programmed or erased. Microchip's memory reference explains these distinctions.
| Term | Meaning |
|---|---|
| Mask ROM | Contents set during manufacture. |
| PROM | Programmable ROM, traditionally programmed once. |
| OTP | One-time-programmable memory. |
| EPROM | Erasable programmable ROM; traditional windowed devices use ultraviolet erasure. |
| EEPROM | Electrically erasable programmable ROM. |
| Flash | Electrically programmable memory erased in blocks; NOR and NAND are major families. |
| SLC, MLC, TLC, QLC | Single-, multi-, triple-, and quad-level cell configurations, distinguished by bits stored per cell. |
| NVRAM | Nonvolatile RAM, a broad category that retains contents without continuous external power. |
Other nonvolatile technologies include MRAM (magnetoresistive RAM), FRAM/FeRAM (ferroelectric RAM), PCM (phase-change memory), and ReRAM (resistive RAM). These describe physical storage mechanisms. Their persistence alone says nothing about whether an AI system learns from an interaction.
GPU memory: VRAM, shared memory, and unified memory
Video RAM (VRAM) is commonly used to mean memory available to a graphics processing unit, or GPU. Depending on the hardware, that can involve dedicated device memory or a shared system-memory arrangement. GDDR and HBM describe memory technologies; VRAM describes a role.
NVIDIA's CUDA guide distinguishes device memory, caches, registers, and programmer-managed shared memory. The names can mislead:
| Term | Meaning |
|---|---|
| Host memory | Memory on the central processing unit (CPU) side of a CPU–GPU application. |
| Device memory | Memory used by the accelerator. |
| Registers | Small, fast storage close to computation. |
| L1, L2, L3 caches | Levels in a processor's memory hierarchy; exact arrangements vary. |
| GPU shared memory | On-chip workspace shared by cooperating threads, not ordinary shared system RAM. |
| Unified memory architecture | CPU and GPU share physical system memory. |
| CUDA Unified Memory | A managed memory abstraction; not a promise of one physical memory pool. |
| Pinned memory | Host memory kept page-locked, often used for device transfers. |
Apple's Metal explanation describes a unified hardware arrangement in which CPU and GPU can access the same memory. That differs from software arranging access or migration across separate memories.
Capacity, bandwidth, and latency are different properties
Capacity measures how much memory is available, commonly in bytes. Bandwidth measures the amount transferred per unit of time. Latency measures how long an access or operation takes. A larger memory pool need not be a faster one. NVIDIA's bandwidth guidance separates theoretical bandwidth from achieved transfers.
Related terms include throughput, the achieved transfer or work rate; memory footprint, the amount used; and working set, the data actively needed. A workload is memory-bound when memory movement limits its performance, rather than arithmetic throughput.
Out-of-memory (OOM) means an allocation cannot be satisfied. It differs from having enough capacity but insufficient bandwidth for the desired speed. Fragmentation can also make allocation difficult.
Framework measurements need interpretation. In PyTorch, allocated memory tracks memory occupied by tensors, while reserved memory includes memory managed by its caching allocator. Unused reserved memory is not automatically a leak. PyTorch's CUDA memory documentation explains the distinction.
Software memory: addresses, lifetimes, and persistence
A program generally works with virtual addresses, which are mapped to physical memory. Virtual memory does not mean unlimited RAM, and it is not another name for swap. The Linux memory-management overview explains address spaces, pages, mappings, and caches.
| Term | Meaning |
|---|---|
| Stack | Storage associated with function calls and local execution state. |
| Heap | Dynamically allocated program memory. |
| Static/global memory | Storage for objects with program-wide or static lifetimes. |
| Pages and page tables | Units of memory mapping and structures that track those mappings. |
| Swap | Secondary storage used to back some memory pages. |
| Memory-mapped file | File contents exposed through a program's address space. |
| Page cache | Cached file data managed by the operating system. |
| Shared/private memory | Mappings with different sharing and isolation behavior. |
| NUMA | Non-uniform memory access: access costs depend on memory location relative to a processor. |
| Copy-on-write | Sharing data until a write requires a private copy. |
Allocation and deallocation, garbage collection, reference counting, ownership, and borrowing concern memory management. Leaks, dangling pointers, and use-after-free concern its correctness. They do not describe an agent's recollection.
Two other distinctions cut across these layers. Volatile/nonvolatile concerns retention without power. Ephemeral/persistent concerns the lifetime a system gives its data. Saving a transcript to an SSD makes it durable; it does not establish how that transcript will influence the next task.
Model execution: weights, activations, and caches
A model occupies more memory than its parameter file alone. Hugging Face's GPU-memory guide distinguishes weights, intermediate activations, gradients, and optimizer state.
| Term | Meaning |
|---|---|
| Weights/parameters | Learned numerical values used by the model. |
| Neural activations | Intermediate values computed while processing an input. |
| Gradients | Derivatives used to update trainable parameters. |
| Optimizer states | Additional values maintained by an optimization algorithm. |
| Temporary workspaces | Buffers needed by computation implementations. |
| KV cache | Stored attention keys and values reused during generation. |
Training and inference have different requirements: ordinary inference does not need the gradients and optimizer state used for training. Batch size, sequence length, numerical precision, and implementation affect the memory footprint.
Quantization represents values at reduced precision. Offloading places some data or computation on another device or storage tier. Sharding partitions it. Activation checkpointing trades recomputation for keeping fewer intermediate activations during training. These are memory-management techniques, not forms of autobiographical recall.
In an autoregressive Transformer, a key–value cache avoids recomputing attention keys and values for preceding tokens. Prefix caching reuses cached computation for a shared input prefix. Neither, by itself, establishes that experience changes future decisions. Hugging Face's cache explanation describes the runtime mechanism.
Neural activation and memory activation are different terms: one names computed tensor values; the other names the availability of remembered information in a context.
Working memory, context windows, and session state
Working memory is a functional workspace for current processing. In ACT-R, buffers expose information from modules; in Soar, working memory represents the current situation as linked objects and attributes. Their structures are not interchangeable.
A model's context window is its supported sequence capacity, usually expressed in tokens. The material supplied within that window can help perform a working-memory role, but the window itself does not specify goals, task updates, or the rules for bringing earlier experience into the task.
Conversation history is a record of messages. Session state can also include outstanding tool calls, selected actions, and application variables. A scratchpad holds intermediate work. The application decides which parts are supplied, retained, summarized, or discarded; ending a user-visible conversation does not prescribe one universal storage behavior.
Compaction summarizes or restructures context to fit a budget. Truncation drops material. A sliding window limits which sequence positions remain available to a particular attention mechanism. Each can affect the evidence available for a response.
A longer context window can hold more material. It does not determine which history deserves attention or how new evidence should revise an earlier conclusion.
Learned knowledge, external records, and retrieval
Parametric memory refers to knowledge encoded in learned model parameters. Pretraining and fine-tuning change those parameters. Adapters, including low-rank adaptation (LoRA), provide a way to train a smaller set of additional parameters, as described in Hugging Face's adapter documentation.
External/non-parametric memory places accessible information outside the model's learned parameters. Documents, databases, vector indexes, knowledge graphs, logs, and transcripts can serve that role. Embeddings represent information for operations such as similarity search; a vector index organizes retrieval.
Retrieval-augmented generation (RAG) uses retrieved information to condition generation. Lewis and colleagues' RAG paper explicitly distinguishes parametric and non-parametric memory. Our RAG comparison examines how retrieval can participate in a broader memory-guided system.
A checkpoint saves model or training state. A snapshot records a state at a point in time. Memorization and generalization describe different ways learned behavior relates to examples; catastrophic forgetting concerns loss of earlier capabilities during later learning. None is equivalent to deleting a database row.
Cognitive and agent memory describe roles
These labels identify what information or learned behavior does, rather than which chip holds it:
| Term | Role |
|---|---|
| Declarative memory | Knowledge represented as facts or events. |
| Semantic memory | Facts and relationships not tied to replaying one particular occasion. |
| Episodic memory | Particular situations represented with their temporal and contextual relationships. |
| Procedural memory | Skills or conditional action knowledge. |
| Associative memory | Recall through relationships among representations. |
| Prospective memory | Retaining an intention for a future action. |
| Short-term/long-term memory | Retention distinctions whose implementation depends on the architecture. |
| Agent/subagent memory | Information and learned patterns available across an agent's work or handoffs. |
| Shared/organizational memory | Experience made available across people, agents, or organizational work. |
The declarative and procedural memory chapter and Soar's semantic and episodic memory chapter explain these distinctions. Agent memory is not one standardized implementation, and calling a log “episodic” does not supply a complete cognitive architecture.
Specialized AI architectures use additional meanings: recurrent hidden state in recurrent neural networks (RNNs) and long short-term memory (LSTM) networks, state-space model state, learned memory slots, differentiable external memory, and associative networks. Experience replay reuses stored examples during learning. These mechanisms belong in the wider vocabulary, but they should not be presented as synonyms for a chat-history store. See the glossary's machine-learning reference.
Emergent Memory: experience changes future influence
In Achiral's canonical framing, Emergent Memory sits between accumulated experience and working memory, shaping what becomes salient and influences subsequent reasoning or action. Organic AI Memory emphasizes formation and adaptation through activity. These are our architectural terms, not names for semiconductor technologies or a universal product specification.
| Mechanism | What changes |
|---|---|
| Formation and encoding | Experience becomes a represented memory. |
| Association | Relationships connect memories with contexts or other memories. |
| Activation and salience | Some remembered information becomes more available or influential now. |
| Reinforcement | Use or feedback changes a memory's later availability or influence. |
| Revision and supersession | New evidence changes an earlier representation or its authority. |
| Decay and forgetting | Availability or influence weakens, or information is suppressed or removed. |
| Consolidation | Experience is reorganized into more durable knowledge or action patterns. |
| Reconstruction | Recall uses retained information and current context to form a representation of the past. |
Reinforcement does not establish truth. Decay does not reliably identify irrelevance. A memory design must preserve important exceptions and respond to corrections, rather than merely favor frequently accessed information.
Consider a hypothetical support agent. A customer explains that a particular recovery method is inaccessible to them. The system records the verified constraint and associates it with that customer's recovery task. On a later case, the constraint becomes available before the agent proposes the same unsuitable method. If the customer corrects it, the next decision must change again.
The record is evidence. The changed availability and action are the behavior to evaluate. Permissions, provenance, retention, and expiry support that behavior; they do not replace it. This example describes a design criterion, not a claim about a deployed Achiral feature.
Questions people ask
Is VRAM the same as AI memory?
VRAM holds data for GPU computation. It can contain weights, activations, or cached attention state. Emergent Memory concerns how experience changes subsequent processing and behavior. A larger GPU memory pool does not establish that capability.
Is a context window the same as working memory?
No. A context window is a model's sequence capacity. Working memory is the role and organization of information used in current processing. An application can use model context as part of its workspace while maintaining goals, pending actions, and other state elsewhere.
Does remembering require changing model weights?
No. Experience can change accessible records, associations, retrieval priorities, or conditional behavior without changing the model's weights. Weight updates are another route to learned change. The evaluation question is what persists from experience and how it affects later behavior.
Are memory bandwidth and capacity interchangeable?
No. Capacity concerns how much fits; bandwidth concerns transfer rate. A workload may fit in memory and still run slowly because moving its data is the bottleneck. Neither measurement describes whether an agent uses its past appropriately.
For the behavioral distinction, continue with Emergent Memory vs Storage Primitive. For individual terms across disciplines, use the Canonical Memory Glossary.
Sources and further reading
- Samsung: DRAM and Microchip: RAM and ROM: semiconductor memory families.
- NVIDIA: CUDA C++ Best Practices: GPU memory hierarchy and transfer performance.
- Apple: Metal compute and unified memory: shared CPU–GPU physical memory.
- Linux: memory-management concepts: virtual memory, mappings, and pages.
- PyTorch: CUDA memory management: allocated and reserved memory.
- Hugging Face: GPU memory usage and caching: model execution state.
- Lewis et al.: Retrieval-Augmented Generation: parametric and non-parametric memory in RAG.
- Achiral: Emergent Memory Systems: the behavioral definition used in this guide.