Skip to main content
Be Bitwise

Floating-Point Numbers

Data Representation · Data & Hashing track

6 min read

Part ofSystems Programming Companion, week 2

In this module

The Problem with Fractions

Integers are great for counting. But the real world is full of measurements that fall between whole numbers: temperature, distance, probability. Computers need a way to represent fractional values, and the dominant solution is floating-point arithmetic, defined by the IEEE 754 standard.

Sign, Exponent, Mantissa

A floating-point number is stored in three parts:

  1. Sign bit - 0 for positive, 1 for negative.
  2. Exponent - controls how large or small the number is (the position of the binary point).
  3. Mantissa (also called the significand) - holds the actual digits of precision.

Think of it like scientific notation. The number 6.022 x 10^23 has a sign (positive), a mantissa (6.022), and an exponent (23). Floating-point does the same dance in base 2.

FormatTotal BitsSignExponentMantissa
Single (float)321823
Double (double)6411152

Two details make the encoding work. The exponent is stored with a bias (127 for float), so a stored exponent of 127 means 2⁰ and 124 means 2⁻³. That lets the hardware compare floats almost as if they were integers. And for normal numbers the mantissa has an implicit leading 1: since every binary number in scientific form starts with a 1, IEEE 754 doesn't bother storing it, which buys one free bit of precision. The catch is that zero has no leading 1, so it needs a special case: an all-zero exponent field switches the implicit 1 off. With a zero mantissa that encodes 0.0 (or −0.0), and with a non-zero mantissa it encodes the tiny subnormal numbers that fill the gap between zero and the smallest normal float.

IEEE 754

IEEE 754 is the standard that defines how floating-point numbers are stored and how arithmetic operations should behave. Nearly every modern processor implements it in hardware, which is why floating-point quirks are consistent across languages and platforms.

Worked Example: Decoding 0.1f

Store 0.1f in a C float and dump its bits. You get 0x3DCCCCCD. Split that into its three fields:

text
0x3DCCCCCD = 0 01111011 10011001100110011001101
s exponent mantissa (23 bits)
  • Sign: 0, so positive.
  • Exponent: 01111011 is 123. Subtract the bias: 123 − 127 = −4. So the number is something times 2⁻⁴.
  • Mantissa: put the implicit 1 back in front and you get 1.10011001100110011001101 in binary, which is about 1.6000000238.

Multiply: 1.6000000238 × 2⁻⁴ = 0.100000001490116. Close to 0.1, but not equal. Notice the repeating 1100 pattern in the mantissa, cut off and rounded up at the end. That rounding is where the error comes from.

Simpler numbers come out exact. 1.0f is 0x3F800000 (sign 0, exponent 127, mantissa all zeros), and -2.0f is 0xC0000000 (sign 1, exponent 128, mantissa zero).

Now take the bits apart yourself. The visualiser starts on 0x3DCCCCCD, and the coloured strips under the bits mark sign, exponent and mantissa. Flip the lowest mantissa bit and see how far the value moves, then load the presets: 2²⁴ shows a mantissa of all zeros, the infinity and NaN presets show what an all-ones exponent means, and the smallest subnormal is a single 1 bit with the exponent field at zero. You can also type a number and see which float it rounds to.

231
230
229
228
227
226
225
224
223
222
221
220
219
218
217
216
215
214
213
212
211
210
29
28
27
26
25
24
23
22
21
20
Decimal1036831949
Hex0x3DCCCCCD
Binary00111101 11001100 11001100 11001101
FieldBitsHexDecimal
sign310x00
exponent (bias 127)30-230x7B123
mantissa22-00x4CCCCD5033165
As float32: 0.1Kind: normalScale: 1.fraction × 2-4 (stored exponent 123)
Exact value stored: 0.100000001490116119384765625

Why 0.1 + 0.2 Does Not Equal 0.3

The fraction 1/10 is clean in decimal. In binary, it's an infinitely repeating sequence, the same way 1/3 (0.333...) repeats forever in decimal. A float has only finitely many mantissa bits, so the representation gets rounded, exactly as you saw above. Both 0.1 and 0.2 pick up tiny rounding errors, and when you add them, those errors compound into something like 0.30000000000000004 instead of exactly 0.3.

Every implementation that follows IEEE 754 gives this answer. You're paying the price of squeezing an infinite fraction into a fixed number of bits, and the price is the same in C, Python and JavaScript.

The practical fix is to stop testing floats for exact equality. Compare against a tolerance instead (fabs(a - b) < 1e-9, with a tolerance chosen for your data).

Predict: float f = 16777216.0f; f += 1.0f; What's in f now?

Still 16777216. That's 2²⁴. A float has 24 bits of precision (23 stored plus the implicit 1), so at this size the gap between neighbouring floats has grown to 2. The true answer, 16777217, sits exactly halfway between two representable values, and the round-to-even rule picks 16777216. Add 1 a million times in a loop and the total never moves.

Precision Limits and Special Values

A 32-bit float gives you roughly 7 decimal digits of precision; a 64-bit double, about 15-16. Need exact decimal arithmetic for financial calculations, say? Floating-point is the wrong hammer. Reach for integer-based representations (storing pence instead of pounds) or a dedicated decimal library.

IEEE 754 also reserves a handful of special values: positive and negative infinity for overflow, and NaN (Not a Number) for things like 0/0. These sentinels let computations limp on without crashing, but they have a nasty habit of propagating silently through your code if you don't watch for them. NaN is even unequal to itself, so x != x is a classic, if slightly baffling, NaN test. Build with -ffast-math and the compiler is allowed to assume NaNs never happen, so it may fold that test to false. GCC does exactly that at -O2 -ffast-math.

Try It: Decode and Encode by Hand

Two by hand, then a prediction. The program at the end checks all of them.

Decode 0x40490FDB into a decimal value.

Split it: 0 10000000 10010010000111111011011. The sign is 0, so positive. The exponent 10000000 is 128, and subtracting the bias leaves 2¹. Put the implicit 1 back and the mantissa is 1.10010010000111111011011, about 1.5707964. Double it and you get 3.14159274, the float closest to π, off by about 9 × 10⁻⁸.

Encode 6.5 as a float. What are its 32 bits in hex?

0x40D00000. 6.5 is 110.1 in binary, or 1.101 × 2². The exponent field stores 2 + 127 = 129, which is 10000001. Drop the implicit 1 and the mantissa is 101 followed by twenty zeros. Together that's 0 10000001 10100000000000000000000, and regrouping into fours gives 0x40D00000.

Predict: for doubles, is 0.1 + 0.2 == 0.3 true? What about 1e16 + 1 == 1e16?

False, then true. The first sum comes out as 0.30000000000000004. The second is the 16777216.0f story again at double width: above 2⁵³ (about 9 × 10¹⁵) neighbouring doubles sit 2 apart, so adding 1 lands exactly halfway and round-to-even sends it back to 1e16.

The program reads a float's bits by memcpy-ing it into a uint32_t. That's the portable way to do it, because casting a float * to a uint32_t * and dereferencing it breaks C's aliasing rules. The last line adds 0.1 ten times and falls just short of 1.

Try itSwap 6.5f for a value of your own, or feed float_from a different bit pattern, and check your hand decoding.
C (GCC 9.2.0)