Skip to main content
Be Bitwise

The Instruction Pipeline

CPU Pipeline & Microarchitecture · CPU Architectures track

5 min read

Part ofSystems Programming Companion, week 6

In this module

Assembly Line for Instructions

Back in CPU Basics you met the fetch-decode-execute cycle: the CPU fetches an instruction, decodes it, executes it, writes the result. In the naive version, each instruction finishes before the next one even starts. Modern CPUs use pipelining to overlap these stages and hammer throughput up dramatically.

The Pipeline Concept

Pipelining slices instruction execution into stages, each running on dedicated hardware. While one instruction sits in execute, the next is being decoded, and a third is being fetched. Think factory assembly line: multiple instructions are in flight at once, each parked at a different stage.

A classic 5-stage pipeline:

Clock12345678
Inst 1IFIDEXMEMWB
Inst 2IFIDEXMEMWB
Inst 3IFIDEXMEMWB
Inst 4IFIDEXMEMWB

Once the pipeline is full (5 cycles for the first instruction to land), one instruction retires every cycle — up to 5x the throughput of the non-pipelined version, in the ideal case with no stalls. Deeper pipelines push clock frequency higher (each stage does less work) at the cost of bigger hazard penalties.

Modern pipelines run far deeper than five stages. Intel's Skylake sits around 14-19; the legendary Pentium 4 Prescott pushed it to 31. Some vendors, Apple among them, don't publish pipeline depths at all, so figures you see quoted for those chips are outside estimates.

Pipeline Hazards

Hazards are moments where the pipeline can't push an instruction to its next stage on the next clock. Three flavours:

Data hazards show up when an instruction needs the result of a previous one that's still in flight:

nasm
add r1, r2, r3 ; r1 = r2 + r3 (result available after EX stage)
sub r4, r1, r5 ; needs r1's new value (in ID stage, r1 not yet written)

Without help, sub reads the stale value of r1. Two ways out: forwarding (route the EX result straight into the next instruction's EX input, skipping WB) or stalling (insert a bubble and wait a cycle).

Try both in the pipeline below. The program is that add/sub pair followed by a load and an instruction that uses the loaded value. Step through it with forwarding off and count the bubbles: there are four, two in front of each dependent instruction. Then tick Forwarding and run it again.

Instructions

Cycle: 1/12500ms
F = FetchD = DecodeE = ExecuteM = MemoryW = WriteBubble/StallHazard
CycleFDEMW
1ADD R1, R2, R3----
2stallADD R1, R2, R3stallstallstall
3stallstallADD R1, R2, R3stallstall
4SUB R4, R1, R5--ADD R1, R2, R3-
5LW R6, 0(R7)SUB R4, R1, R5--ADD R1, R2, R3
6stallLW R6, 0(R7)SUB R4, R1, R5stallstall
7stallstallLW R6, 0(R7)SUB R4, R1, R5stall
8ADD R8, R6, R4--LW R6, 0(R7)SUB R4, R1, R5
9-ADD R8, R6, R4--LW R6, 0(R7)
10--ADD R8, R6, R4--
11---ADD R8, R6, R4-
12----ADD R8, R6, R4

Detected RAW Hazards

#2 SUB R4, R1, R5 reads R1 written by #1 ADD R1, R2, R3 -> 2 stalls
#4 ADD R8, R6, R4 reads R6 written by #3 LW R6, 0(R7) -> 2 stalls
#4 ADD R8, R6, R4 reads R4 written by #2 SUB R4, R1, R5 -> 1 stall
Instructions: 4Total cycles: 12Stall cycles: 4CPI: 3.00

Forwarding erases the add/sub stall completely, but one bubble survives in front of the last ADD. That's a load-use hazard, and no amount of forwarding wires can remove it. An ALU result exists at the end of EX, just in time to be forwarded into the next instruction's EX. A loaded value doesn't exist until the end of MEM, a full stage later, by which point the dependent instruction would already be in EX:

Compilers try to hide this by scheduling an independent instruction into the slot right after a load.

Control hazards crop up at branches. The pipeline has already fetched instructions after the branch but has no idea whether the branch will be taken. If it is, those fetched instructions are wrong and have to be tossed. This is where branch prediction (next lesson) earns its keep.

Structural hazards appear when two instructions fight over the same piece of hardware — both wanting the memory port, for example. The fix is duplicating the resource or scheduling around the conflict.

Pipeline Stalls and Bubbles

When forwarding can't paper over a hazard, the pipeline injects a bubble: a NOP-shaped placeholder that drifts through stages, taking up space and doing nothing. Each bubble is one cycle of throughput burned. Stalls cascade upstream: if EX stalls, ID stalls (can't send a new instruction into a busy EX), and IF stalls too. Everything behind the stall freezes.

Data Hazard Types

Data hazards are sorted by the order of reads and writes:

  • RAW (Read After Write): the true dependency. Instruction B reads a register that A writes. B has to wait for A's result. The most common hazard you'll see.
  • WAR (Write After Read): B writes a register that A reads. Let B execute before A reads, and A gets the wrong value. Can't happen in an in-order pipeline (A is always ahead of B), but very much a concern for out-of-order execution.
  • WAW (Write After Write): both instructions write the same register. They have to write in program order. Again, mainly an out-of-order problem.

Register renaming (Lesson 3) wipes WAR and WAW hazards out entirely by handing each write its own physical register.

Modern Pipeline Features

Real pipelines have nothing on the tidy 5-stage model. Some of what the textbook leaves out:

  • Multiple decode stages: x86 instructions are 1 to 15 bytes long. Parsing the variable-length encoding and finding the boundaries takes several stages.
  • Micro-op decomposition: hairy CISC instructions are chopped into simpler micro-operations (uops) that flow through the pipeline like RISC.
  • Pipeline width: modern CPUs decode, rename, and dispatch 4-6+ instructions per cycle (superscalar, Lesson 4).
  • Out-of-order execution: instructions fire when their operands are ready, not in program order (Lesson 3).

The pipeline is the substrate everything else stands on. Every microarchitectural trick — branch prediction, out-of-order execution, caching — exists to keep the pipeline fed and flowing.