Skip to main content
Be Bitwise

Signed vs. Unsigned Confusion

Integer Overflows & Type Confusion · Exploitation Fundamentals track

5 min read

Part ofSystems Programming Companion, week 1

In this module

The Same Bits, Two Meanings

In Binary & Number Systems, you learned about two's complement - the system computers use to represent negative numbers. Here is the critical security implication: the same bit pattern means completely different things depending on whether the program interprets it as signed or unsigned.

The byte 0xFF is 255 as unsigned. It's -1 as signed two's complement. The bits in memory are identical. Only the interpretation changes — and that difference can be catastrophic.

Both widgets below start on 0xFF. The first reads the byte as unsigned, the second as signed:

27
26
25
24
23
22
21
20
Decimal255
Hex0xFF
Binary11111111
27
26
25
24
23
22
21
20
Decimal-1
Hex0xFF
Binary11111111

Click bits in each to make them match, then watch the decimal readouts. They agree as long as the top bit is 0. Set bit 7 and the signed one drops by 256, which is the whole bug class in one switch.

How Confusion Arises

Consider a function that checks whether a user-provided length is safe:

c
int length = get_user_input(); // Signed integer
if (length > MAX_BUFFER_SIZE) { // Signed comparison
return ERROR;
}
memcpy(buffer, data, length); // length cast to unsigned size_t

An attacker sends -1 as the length. The signed comparison -1 > 1024 is false, so the check passes. But memcpy takes its length as a size_t, which is unsigned, so the same -1 turns into the largest value a size_t can hold: 4,294,967,295 on a 32-bit system, or about 18.4 quintillion on a 64-bit one. The copy tries to move an absurd amount of data, overflowing the buffer or crashing the program.

Implicit Conversion

In C and C++, the compiler silently converts between signed and unsigned types in many situations - comparisons, function arguments, arithmetic with mixed types. These implicit conversions happen without any warning by default, making signed/unsigned bugs easy to introduce and hard to spot.

Worked Example: C's Conversion Rules in Action

The rule behind most of these bugs is short. When an operation mixes a signed and an unsigned integer of the same width, C converts the signed one to unsigned before doing anything. A negative number becomes a huge positive one, by adding 2³² for a 32-bit int.

Try this:

c
unsigned int a = 1;
int b = -1;
if (b < a)
puts("less");
else
puts("not less");

It prints "not less". Before comparing, b is converted to unsigned int, and -1 becomes 4,294,967,295, which is certainly not less than 1. Compile with -Wall -Wextra and gcc warns about comparing integers of different signedness. Without those flags, it says nothing.

sizeof is a frequent trigger, because it returns a size_t. Write if (len < sizeof(buf)) with an int len, and len is converted to size_t first. Here that happens to be safe, since a negative len now fails the check. But rearrange the same idea as if (sizeof(buf) - len >= 0) and the result of the subtraction is unsigned, which can never be negative, so the check passes whatever len is.

Predict: char c = 0xFF; if (c == 0xFF) puts("match"); Does it print "match"?

It depends on the CPU. On x86, plain char is signed, so c holds -1. In the comparison, c is promoted to the int -1, while 0xFF is the int 255, and they don't match. On ARM Linux, plain char is unsigned, c holds 255, and it prints "match". Same source code, different behaviour, which is exactly the portability trap the next section describes.

Real-World Examples

This class of bug has appeared throughout computing history. A classic example: a network protocol specifies a packet length as a 16-bit signed integer. The receiving code checks that the length is positive and not too large, then passes it to a memory allocation function that treats it as unsigned. Sending a carefully chosen negative value passes the safety check but causes the allocator to see a different, exploitable size.

Another common pattern appears in loop bounds. A signed loop counter compared against an unsigned array length can cause the loop to execute far more iterations than intended if the counter goes negative.

Why C Makes This Worse

C was designed for efficiency, not type safety. The language freely converts between signed and unsigned integers, often silently. Different platforms use different sizes for the same type; a char might be signed on one architecture and unsigned on another. Code that's been working correctly for years can harbour a signed/unsigned bug that only surfaces on a different target.

Defensive Thinking

Use unsigned types consistently for sizes, lengths, and indices; size_t exists for exactly this. When you must convert between signed and unsigned, do so explicitly and check the value first. For the example at the top, that means rejecting negatives before anything else:

c
if (length < 0 || (size_t)length > MAX_BUFFER_SIZE)
return ERROR;

Turn on -Wsign-compare and -Wsign-conversion and treat the warnings as errors. They're noisy on old code, and every one of them is worth a look.