Skip to main content
Be Bitwise
CWhat Your C Code Actually Does

Part ofSystems Programming Companion, week 4

Function Calls Under the Hood

Every C function call kicks off a small choreography: pushing data onto the stack, jumping to fresh code, eventually unwinding the lot to come back. This lesson traces every beat of it, from call to return.

How main() Gets Called

Reasonable assumption: main() is where your program starts. Wrong. The real entry point is _start, supplied by the C runtime (crt0). Here's the sequence of events before main ever sees a register:

  1. The OS loader maps your executable into memory
  2. Control jumps to _start
  3. _start calls __libc_start_main
  4. The runtime initializes the heap, sets up stdio buffers, and runs global constructors
  5. Finally, main(argc, argv) is called

Pop open the disassembly of any compiled C program and you'll see it:

bash
objdump -d program | grep -A 20 "<_start>"
asm
_start:
xor ebp, ebp ; Clear the frame pointer: marks the outermost frame
mov r9, rdx ; Loader's cleanup function (rtld_fini), 6th argument
pop rsi ; argc, straight off the stack, 2nd argument
mov rdx, rsp ; argv now sits at the top of the stack, 3rd argument
and rsp, -16 ; Realign the stack to 16 bytes before the call
push rax ; Padding, keeps the alignment
push rsp ; Stack end, 7th argument
xor r8d, r8d ; fini (unused since glibc 2.34)
xor ecx, ecx ; init (unused since glibc 2.34)
mov rdi, QWORD PTR [rip+0x2fd8] ; Address of main, 1st argument
call QWORD PTR [rip+0x2fd2] ; __libc_start_main(main, argc, argv, ...)
hlt ; Should never reach here

This is glibc's _start for x86-64 (sysdeps/x86_64/start.S), and the offsets in the last two loads will differ in your binary. The first argument to __libc_start_main is main itself, a function pointer, which is how the runtime knows what to call once setup is done. At entry the kernel left argc at the top of the stack with the argv pointers right above it, so a single pop both fetches argc and leaves rsp pointing at argv. It's a neat trick, and a good first reminder that the calling convention below applies to calls between functions. The kernel's handover to _start follows its own rules.

The System V AMD64 Calling Convention

On x86-64 Linux (and macOS), function calls obey the System V AMD64 ABI — a shared contract every piece of compiled code on the system signs up to. The rules:

Integer/pointer arguments are passed in registers, in this order:

  1. rdi - first argument
  2. rsi - second argument
  3. rdx - third argument
  4. rcx - fourth argument
  5. r8 - fifth argument
  6. r9 - sixth argument

More than six arguments? Anything past the sixth spills onto the stack.

Return values come back in rax (or rax:rdx for 128-bit values).

Caller-saved registers (the caller must save these if it needs them after the call): rax, rcx, rdx, rsi, rdi, r8, r9, r10, r11.

Callee-saved registers (the called function must restore these before returning): rbx, rbp, r12, r13, r14, r15.

Watch it play out:

c
int compute(int a, int b, int c) {
return a * b + c;
}
int main() {
int result = compute(3, 4, 5);
return result;
}

Compiled with gcc -S -masm=intel -O0:

asm
compute:
push rbp ; Save caller's base pointer
mov rbp, rsp ; Establish our stack frame
mov DWORD PTR [rbp-4], edi ; a (from rdi) -> stack
mov DWORD PTR [rbp-8], esi ; b (from rsi) -> stack
mov DWORD PTR [rbp-12], edx ; c (from rdx) -> stack
mov eax, DWORD PTR [rbp-4] ; Load a
imul eax, DWORD PTR [rbp-8] ; eax = a * b
add eax, DWORD PTR [rbp-12] ; eax = a * b + c
pop rbp ; Restore caller's base pointer
ret ; Return with result in eax
main:
push rbp
mov rbp, rsp
sub rsp, 16 ; Reserve stack space for locals
mov edx, 5 ; Third arg (c) -> edx
mov esi, 4 ; Second arg (b) -> esi
mov edi, 3 ; First arg (a) -> edi
call compute ; Push return address, jump to compute
mov DWORD PTR [rbp-4], eax ; Store result from eax
mov eax, DWORD PTR [rbp-4] ; Load result for return
leave ; Equivalent to: mov rsp, rbp; pop rbp
ret

Anatomy of a Stack Frame

Here's compute's frame just after the three movs that spill its arguments. Row numbers are byte offsets counted up from rbp-16, so rbp itself sits at 10. The blue band (the visualiser calls it a buffer) is the 16 bytes below rbp: c = 5 at rbp-12, b = 4 at rbp-8, a = 3 at rbp-4, each stored little-endian, with four unused bytes at the bottom. Above it sit the 8 bytes of saved rbp and then the return address that call pushed. Only its lowest byte is drawn, and both saved values are illustrative because they change from run to run.

Stack Frame View

0000Buffer
0100Buffer
0200Buffer
0300Buffer
0405Buffer
0500Buffer
0600Buffer
0700Buffer
0804Buffer
0900Buffer
0A00Buffer
0B00Buffer
0C03Buffer
0D00Buffer
0E00Buffer
0F00Buffer
10E0Saved rbp← SP
11DFSaved rbp
12FFSaved rbp
13FFSaved rbp
14FESaved rbp
157FSaved rbp
1600Saved rbp
1700Saved rbp
1856Return addr
1911Return addr
1A00Return addr
1B00Return addr
1C00Return addr
1D00Return addr
1E00Return addr
1F00Return addr
Regions
Buffer16 B
Saved rbp8 B
0x00007ffeffffdfe0
Return addr8 B
0x0000000000001156
Overflow direction
Buffer→Saved rbp→Return addr

rsp points at the saved rbp, the same place as rbp, because compute never runs sub rsp. It calls nothing, so the ABI's 128-byte red zone below rsp is safe scratch space for its locals.

The function prologue (push rbp; mov rbp, rsp) builds the frame. The epilogue (pop rbp; ret) demolishes it.

call does two things in one shot:

  1. Pushes the address of the next instruction (the return address) onto the stack
  2. Jumps to the target function

ret runs the same trick in reverse:

  1. Pops the return address off the stack
  2. Jumps there

More Than Six Arguments

So what about more than six integer arguments?

c
int many_args(int a, int b, int c, int d, int e, int f, int g, int h) {
return a + b + c + d + e + f + g + h;
}
int main() {
return many_args(1, 2, 3, 4, 5, 6, 7, 8);
}
asm
main:
push rbp
mov rbp, rsp
push 8 ; 8th arg goes on stack (pushed first!)
push 7 ; 7th arg goes on stack
mov r9d, 6 ; 6th arg -> r9
mov r8d, 5 ; 5th arg -> r8
mov ecx, 4 ; 4th arg -> ecx
mov edx, 3 ; 3rd arg -> edx
mov esi, 2 ; 2nd arg -> esi
mov edi, 1 ; 1st arg -> edi
call many_args
add rsp, 16 ; Clean up the 2 pushed args (8 bytes each)
leave
ret

Arguments 7 and 8 hit the stack backwards — 8 first, then 7. That ordering puts them in the right slot when the callee reads memory low-to-high. The first six rides registers as before.

With nine arguments, which ones does the callee find above its saved rbp (in the caller's frame) and which below it? Make a guess, then check:

Try itPredict the sign of each offset, run it, then add a tenth argument and guess where it lands.
C (GCC 9.2.0)

g, h and i show up at rbp+16, +24 and +32: past the saved rbp at +0 and the return address at +8, one 8-byte slot each, even for an int. The first six have negative offsets because at -O0 the function copies each register into its own frame so it has an address to give you. The exact negative numbers depend on the compiler.

The cdecl Convention (32-bit)

Back on 32-bit x86, the older cdecl convention parked every argument on the stack:

asm
; 32-bit cdecl: calling add(3, 4)
push 4 ; Second argument pushed first
push 3 ; First argument pushed second
call add
add esp, 8 ; Caller cleans up the stack

Every argument costs a memory write on the way in (push) and a memory read once inside the function. System V keeps the first six in registers instead. On a modern CPU the gap is smaller than it sounds, because store-to-load forwarding hands a freshly pushed value straight to the load that reads it, but it's still extra instructions and extra memory traffic on every call.

Return Values

Simple types come back through rax:

c
long get_value() {
return 42;
}
asm
get_value:
mov eax, 42 ; Return value in eax (zero-extends to rax)
ret

For 128-bit values — say, a struct holding two 64-bit fields — the compiler splits it across rax and rdx:

c
struct pair { long a; long b; };
struct pair make_pair() {
struct pair p = {10, 20};
return p;
}
asm
make_pair:
mov rax, 10 ; First 8 bytes of return value
mov rdx, 20 ; Second 8 bytes of return value
ret

Bigger structs trigger a quiet sleight of hand: the caller slips in a pointer as an invisible first argument, and the function fills the return value through it.

Tail Calls

A tail call is just any call sitting in the very last slot of a function's body:

c
int factorial_helper(int n, int acc) {
if (n <= 1) return acc;
return factorial_helper(n - 1, acc * n); // Tail call
}

Turn optimisation on and the compiler swaps the recursive call for a jmp, recycling the current stack frame:

asm
factorial_helper:
cmp edi, 1
jle .return_acc
imul esi, edi ; acc = acc * n
dec edi ; n = n - 1
jmp factorial_helper ; Jump, not call! No new stack frame
.return_acc:
mov eax, esi
ret

That's tail call optimisation (TCO). Recursion turns into a loop and stack usage stays flat no matter how deep the call chain pretends to go. Not every compiler promises TCO, but GCC and Clang deliver it reliably at -O2 or higher.

What You Should Take Away

Every function call costs you something measurable:

  • Pushing and popping the stack frame (2-4 instructions)
  • Saving and restoring callee-saved registers (when they get touched)
  • The call and ret instructions themselves
  • Possible cache misses if the callee's code isn't already sitting in the instruction cache

Inside a hot loop those costs add up fast. Hence why compilers love inlining small functions — drop the call entirely, paste the function body straight into the caller, done.

Calling conventions also explain a lot of seemingly unrelated mysteries: why variadic functions like printf work, why function pointers carry the signatures they do, and how debuggers reconstruct a stack trace. With frame pointers that's just walking the chain of saved rbp values back through every frame. Optimised x86-64 code usually omits them (-O2 implies -fomit-frame-pointer), so debuggers, profilers and C++ exception handling fall back on the unwind tables the compiler writes into .eh_frame (DWARF call-frame information), which describe how to find each caller's frame from any instruction address.