Skip to main content
Be Bitwise

Demand Paging & Copy-on-Write

Virtual Memory Internals · OS Core Internals track

5 min read

Part ofSystems Programming Companion, week 10

In this module

Lazy Is Efficient

Modern operating systems don't eagerly allocate physical memory the moment a process asks for it. They use demand paging: physical pages are only allocated when the process actually touches them for the first time. That laziness is a deliberate optimisation, and it saves staggering amounts of memory.

How Demand Paging Works

When a process calls malloc() or mmap() to grab memory, the kernel updates the page tables to mark the new virtual address range as valid but sets the Present bit to zero in each page table entry. No physical memory is consumed yet.

The first time the process reads or writes one of those pages, the CPU triggers a page fault. The kernel's fault handler recognises this as a valid access to a lazily-allocated page, allocates a physical frame, zeroes it out (so you can't peek at another process's old data), maps it into the page tables, and returns. The process resumes none the wiser.

Key Concept

Demand paging holds off physical memory allocation until the first access. A process can reserve far more virtual memory than the system has physical RAM. Only the pages actually touched consume real resources.

That's why a process can call malloc(1 GB) and succeed even on a 512 MB system, as long as it doesn't actually touch all of that memory. Whether the kernel agrees depends on the vm.overcommit_memory sysctl. Mode 1 grants every request. Mode 2 enforces a strict commit limit. The default, mode 0, is a heuristic that refuses only obvious overcommits: a single request larger than total RAM plus swap fails, so on a 512 MB machine the 1 GB call succeeds only if there's enough swap (or the mode is 1).

Copy-on-Write (CoW)

The fork() system call creates a child process that's a copy of the parent. Naively cloning the parent's entire address space would be slow and wasteful, especially since the child often turns around and calls exec() immediately, throwing all that work away.

Instead, the kernel uses Copy-on-Write:

  1. At fork(), the kernel doesn't copy any pages. It marks all of the parent's writable pages as read-only in both the parent's and child's page tables.
  2. Both processes share the same physical pages.
  3. When either process tries to write to a shared page, the CPU raises a page fault (the page is read-only, after all).
  4. The kernel's fault handler recognises this as a CoW fault, allocates a new physical page, copies the contents of the original page into it, maps the new page as writable into the faulting process's page table, and returns.

The beauty of CoW: pages that are never written, like read-only code pages, are never copied. A fork() followed by exec() might copy only a handful of pages instead of the entire address space.

c
pid_t pid = fork();
// At this point, parent and child share all pages (read-only)
if (pid == 0) {
// Child process
global_var = 99;
// Writing triggers a CoW fault - child gets its own copy
// Parent's global_var is unchanged
}

Page Replacement

Physical memory is finite. When the system runs low on free frames, the kernel has to reclaim pages from running processes. The page replacement algorithm decides which pages to evict.

Pages fall into two categories:

  • Clean pages (unmodified since loaded from disk) can be discarded immediately. The file still has the data if it's needed again.
  • Dirty pages (modified) must be written out to swap or the backing file before their physical frame can be reused.

Linux runs an approximation of Least Recently Used (LRU) with two lists: an active list and an inactive list. Pages get promoted to the active list when accessed and demoted to the inactive list when they go unused. Pages at the tail of the inactive list are first in line for eviction. Since Linux 6.1 there's also an optional replacement, the multi-generational LRU (MGLRU), which sorts pages into several age generations instead of two lists; some distributions enable it by default.

The OOM Killer

When the system is completely out of memory and page replacement can't free enough pages, Linux invokes the Out-of-Memory (OOM) killer. It picks a process to terminate based on a scoring heuristic, generally going after the biggest memory hog while sparing the system processes you can't live without.

bash
# Check a process's OOM score (higher = more likely to be killed)
cat /proc/<pid>/oom_score
# Protect a critical process from the OOM killer
echo -1000 > /proc/<pid>/oom_score_adj

Why It Matters

Demand paging and CoW are more than performance optimisations. They're security-relevant mechanisms. CoW faults must be handled correctly or they become vulnerabilities. The infamous Dirty COW exploit (CVE-2016-5195) was a race condition in the kernel's CoW page fault handler that let unprivileged users write to read-only files. Understanding these mechanisms at the page-table level is essential for defending and attacking systems alike.