Posted on Thu 06 October 2011 in Opinion

A colleague made me aware of this post by Ryan Dahl, the author of Node.js, where he complains that most software is overly complicated, that the whole software situation is messed up and that all that matters is the end user experience. In a response to his own post, he openly admits that Node.js is also too complex. But what turns out to be interesting is that his post contains a rant about the complexity of Unix and the software built on top of it.

This guy took offense by the Unix rant and calls Node.js a cancer, and goes on to say that the seemingly all-accepted premise that asynchronous I/O and an event loop saves the day is not true. (I think he missed the point of the original post, but took the chance to release some steam!) Anyway, to make his point he writes a tiny Node.js server that computes a Fibonacci number in a CPU-intensive way using a simple recursive algorithm, demonstrating the possibility to “block” even with Node.js (though technically, blocking is probably the wrong term).

Finally, someone created node-fib to show that it’s indeed possibly to calculate a Fibonacci number recursively using Node.js without “blocking”.

All of this got me thinking. First off, I too believe that much software is overly complicated, and that any software program of age is bound to contain a lot of accidental complexity. While this can be caused by bad decision-making, sloppy coding or over-ambition, it’s also the case that many decisions were probably sound in their context and it’s only over time that they have become obsolete and in retrospect diverging (the saying “the road to hell is paved with good intentions” comes to mind, although it may be a bit harsh). Can we make perfect software from the beginning? Of course not! We have to accept that we learn over time, and what we can do is to be agile:

  • Abandon bad ideas and bad designs as soon as we realize that they are bad.
  • Write simple code that doesn’t solve problems of the future (I think it’s ok to plan ahead though, but what that means is a different discussion).
  • Refactor the code as soon as we detect problems with it.

Of course, software that has been released is not always easy to change, since we have to assume that users now depend on all APIs, good or bad. But if we keep it simple, our APIs hopefully won’t be that bad.

On to Node.js being a cancer. Is that true? I don’t think so, but perhaps it has been over-used, over-exposed or over-something… I’ve used the Twisted framework for Python a bit, and I like asynchronous I/O and event loops! I find that thinking asynchronously results in a nice mental model of the software I’m writing. It’s almost declarative! But I know that even if I have a hammer, not everything is a nail! There are certain problems that lend themselves well to being solved in an asynchronous way, and there are problems that do not. It’s always possible to solve a problem using the wrong solution, and I think the key is to constantly question what your’re doing.

Finally, node-fib. Well, it’s just messy! With the algorithm intertwined with the asynchronous infrastructure, the readability is drastically lowered. And if the readability of code is low, the code is not simple. Period! But what I find interesting about node-fib is the class of problems it tries to solve, namely to instrument the code that performs a lengthy operation in a way that disrupts the atomicity of the operation. Node-fib chops the algorithm into pieces so that it can be executed a little at a time. In one project, I instrumented an algorithm with progress reporting to ultimately be able to show a progress bar in a GUI. In both cases, the code that contains the algorithm is no longer “clean” and easily understandable. Its simplicity has been reduced through the introduction of dependencies on infrastructure. Apart from Aspect-Oriented Programming, which is only applicable in some specific situations and has its own problemschallenges, I don’t know of anything that lets you keep the algorithm entirely intact and infrastructure-free. As with everything else, it’s a trade-off!