About those JavaScript performance tests

In my last couple of articles, I wrote about some simple performance tests to compare Array.some with a for loop. I noticed that the for loop ran substantially faster the second time it was called in my process, while Array.some did not.

This got me curious. I wanted to know what was happening, so I dug around a little bit. It’s the kind of behavior that can be pretty tough to track down. I started with freenode for #Node.js. I described what I’d seen and got some feedback. I didn’t find a definitive answer, although I did get a clue. I was told essentially that “microbenchmarking” is pointless, and one should just write maintainable code, and let the JIT compiler worry about performance. I was directed to Mr Aleph‘s website. Mr Aleph is a compiler engineer and enthusiast, and I found a lot of interesting articles at his site.

In 2012 already, he had written a microbenchmarks fairy tale. It turns out that the V8 JavaScript engine compiler performs various optimizations that can produce misleading benchmarks. As an example, he talks about LICM (“loop-invariant code motion”). Oh god, the jargon! But he gives an example, which I’ll explain now.

Prior to working with JavaScript, I was a Java coder. In Java, one commonly writes for loops like this: for (int i = 0; i < arr.length; i++) {...}.

When I was getting started using JavaScript, I was told to assign Array.length to a variable outside the loop, and then reference the variable in the iteration, like this:

var len = arr.length;
for (var i = 0; i < len; i++) {
    ...
}

The reason given was that JavaScript is an interpreted language, and it will evaluate Array.length on every iteration over the loop! Sounds wasteful to me. So I’ve been dutifully, manually setting a variable to Array.length outside my loop ever since then.

Java developers don’t need to do this, because Java is a compiled language. The compiler performs optimizations behind the scenes which basically do what I was doing manually in JavaScript.

Flash forward to 2008-ish, about 10 years ago, when Google Chrome was launched, integrated with the V8 engine. If you thought I’d gotten off-track, I’m now back on it. According to Mr. Aleph, the V8 engine uses LICM, and LICM performs the optimization I’ve been doing manually in assigning Array.length to a variable outside the loop. And that’s just the tip of the optimization iceberg.

If you want to read more about the V8 engine, there’s an excellent series about it written by Alexander Zlatkov in 2017. Start with How JavaScript works: inside the V8 engine + 5 tips on how to write optimized code. You should also take a good look at Mr. Aleph‘s articles.

Once JavaScript is not strictly being interpreted, trying to benchmark and optimize your code manually is probably just a mistake. Mr. Aleph’s conclusion, in 2012, was

The time when you could have easily guessed the cost of a single operation by simple observation is over. If you want to know something then sometimes the only way is to go and actually learn it from the inside out.

Mr. Aleph, “microbenchmarks fairy tale“, 2012

My own conclusion is that you should not think you are going to outsmart an optimizing compiler. Just write code which is readable. If, at some point, you notice bottlenecks, that’s when it makes sense to dig for performance improvements.

But I’ll still move Array.length outside of my iterators 😉

If you found this interesting, click the subscribe button below! I write a new post about once a week.

Leave a Reply

Your email address will not be published. Required fields are marked *