JVM JIT — When the Compiler Removes Your Code

I had a post a few years back about the JIT performing dead code elimination on my benchmarks — that moment where I was timing a loop and the execution time came back as 0 milliseconds because the JIT had looked at my test, decided the result was never used, and simply removed the code.

That post was more about the surprise than the explanation. This one goes deeper into what the JIT actually does, because it has very practical implications once you start doing performance work or trying to understand why a method in production is faster than it looks like it should be.

What the JIT does

When the JVM starts, it interprets bytecode. After a method has been called enough times — the default threshold is around 10,000 invocations — the JIT compiles it to native machine code. And while doing that, it applies optimisations that a static compiler can’t, because it has runtime information.

It knows which code paths are actually being taken.

A static compiler has to be conservative. The JIT can be aggressive.

Dead code elimination

The one that caught me out:

// This entire loop may be eliminated
for (int i = 0; i < 1_000_000; i++) {
    Math.sqrt(i);   // result discarded — JIT removes the call
}

The fix is to consume the result:

double result = 0;
for (int i = 0; i < 1_000_000; i++) {
    result += Math.sqrt(i);
}
System.out.println(result);

If you’re doing serious Java benchmarking, use JMH. It handles this via Blackhole.consume() — a method that tricks the JIT into thinking the result is used without actually printing it on every iteration.

Inlining

Probably the JIT’s most impactful optimisation. When a method is small enough (roughly 35 bytecodes), the JIT replaces the call site with the method body directly. No call overhead, and the inlined code becomes eligible for further optimisations in the calling context.

This is why Java getter/setter patterns perform well at runtime despite looking like overhead at the source level — the getters get inlined away.

# See inlining decisions
$ java -XX:+PrintCompilation -XX:+UnlockDiagnosticVMOptions \
       -XX:+PrintInlining -jar myapp.jar 2>&1 | grep -i inline

Speculative optimisations

This is where it gets interesting. The JIT makes assumptions based on what it’s observed so far and generates optimistic native code. If those assumptions turn out to be wrong, it deoptimises — discards the compiled code, falls back to interpretation, then recompiles with updated information.

Example: a method that has always received ArrayList objects gets compiled with a direct ArrayList.size() call (no virtual dispatch overhead). Then someone passes a LinkedList. Deoptimisation happens, recompilation happens with a proper virtual call.

This is mostly fine. But if it happens constantly — because you have genuinely polymorphic code — it’s a performance drain. You can observe it:

$ java -XX:+PrintCompilation -jar myapp.jar | grep "made not entrant"

Frequent “made not entrant” entries on the same method is worth investigating.

What this means for startup

On a freshly started JVM, everything runs interpreted. The hot paths haven’t been compiled yet. This matters for autoscaling — a new Kubernetes pod won’t perform as well as an established one until the JIT has had time to warm up. If you’re autoscaling aggressively and routing traffic to new pods immediately, the first few users to hit a fresh pod get slower responses.

It also matters for benchmarking. Always warm up before measuring. Let the JIT compile the hot paths, then start recording results.

Tiered compilation

Modern JVMs use tiered compilation (default since JDK 8):

• Interpreter (cold code)
• Client compiler (C1) — quick to compile, moderate optimisations, profiles the code
• Server compiler (C2) — takes longer, heavily optimised native code using the profiling data

Code moves up the tiers as it accumulates invocations. C2 is where you get the aggressive inlining, loop unrolling, and vectorisation that make Java fast at steady state.

For very short-lived processes (CLI tools, functions), most code never reaches C2. This is part of the appeal of GraalVM native image — compile everything ahead of time at the cost of some dynamic optimisations.