TDD and prototyping—a love story?

Can you combine Test-Driven Development (TDD) with prototyping? How can you write tests for functionality that is largely unknown? Where do you even start if you’re on unfamiliar ground and you need to explore to learn about possibilities and constraints? How can you write tests when you don’t know which code you need to write?

These are questions I often hear from people who don’t regularly practice TDD. Sometimes rhetorically as an excuse to stay away from TDD, but taken seriously the questions are very relevant and interesting to discuss! Due to the nature of software, there will always be unknowns. Software development is an activity where all stakeholders learn more and more about the domain and infrastructure as time passes. It is important to be able to work systematically in such an environment.

People who work with me know that I think TDD is extremely important (though they’d probably use stronger words). Not in a zealous way, because TDD is just a means to an end (it’s in the name—it Drives Development). But I do believe it to be one of the most important tools a software developer can use for producing high-quality code.

I rarely deploy untested code, but if I do I always get a feeling that bad things will happen. Something like this:

So yeah, I use TDD a lot, and I really like the level of control it gives me and my colleagues who work with (or will work with) the code.

Topic for an upcoming blog post: How TDD makes you a better team player!

That said, I also from time to time end up in a situation where I need to do exploratory development. It can be that I need to integrate with a service that is sparsely documented, or that I need to develop a new feature but it’s not obvious where in the stack it should live. In such cases, using TDD can be non-trivial.

Here’s what I usually do!

Production-ready prototype

I don’t do this. I just want to mention it as a contrast to the other things I talk about.

Production-ready prototype is an oxymoron. A prototype is by definition something that is only ever a first version and never the final version. (Yes, software is never “final,” but here I mean final for production.)

I see direct productization of prototypes as an anti pattern that typically happens when you create a prototype that works sufficiently well that people around you (*cough* project managers *cough*) mark the feature as complete and decide to ship it.

This is bad. A prototype needs to be formalized in some way. When a prototype is formalized, it stops being a prototype and instead becomes production-ready code.

Throw-away prototyping

Actually, I do this a lot less than I’d like to. With throw-away prototyping, you build a prototype as a pure learning exercise, then throw the prototype away. There’s no need to use TDD, because the resulting code will never see a production server.

The beauty of throw-away prototyping is that mental constraints (resulting from ideas about production quality) disappear, but first and foremost that the second time you do something it usually becomes a lot better than the first time! If you have ever lost your code in a computer crash, you’re well aware of this.

After throwing away the prototype, there’s no excuse for not using TDD to build the feature/integration/whatever for real. Most—if not all—unknowns should be gone. If they are not, perhaps more prototyping is needed.

Throw-away prototyping is an excellent way to formalize a prototype, though the basis for formalization is the learnings achieved through the prototyping activity rather than the prototype code itself.

High-level blackbox test

I do this a lot. Regardless of the nature of the thing I’m going to build, and regardless of the amount of exploration needed, I always write one high-level test. Some important characteristics of such a test are:

• It must be completely implementation independent (which follows from it being blackbox, but it’s worth pointing out).
• It should encode a central and representative happy path. Failure modes and edge cases come later.
• It doesn’t need to contain an actual assertion. It’s perfectly fine for it to just produce console output, such as performance numbers or the transformed/processed response from a service call.
• It is in itself a prototype, meaning that it will not remain in the test suite after prototype formalization.

To me, such a test serves three very important purposes:

1. It keeps me on track and in some way indicates when I have reached my initial goal (even if that goal might be a bit lofty, e.g. when I do performance optimizations).
2. It allows me to avoid the slow and tedious run-click-crash¹ approach. A test is quick to run and makes development a lot more pleasant!
3. It demonstrates some degree of testability, which is important when the prototype will be formalized later.

As an example, I recently did some robustness improvements in code that talks to a Cassandra database. I had no idea which exceptions I would get in various problematic situations (such as network outage) and I only had a vague idea about how a connection problem would affect the existing operation-level retry behavior. I ended up writing a test that fired off a steady stream of operations against the storage facade and in the end printed some metrics and verified that all operations went through. As an added bonus, writing that test forced me to think about the best way to simulate load on the component in question.

Can you use a small program (“main method”) instead of a test? Sure, absolutely! A benefit with a test, however, is that it lives in a test context, which means that all the constraints (test dependencies, test data, etc.) that inevitably will apply later on are present already.

It should be noted that a high-level blackbox test is very useful also if the prototype is meant to be thrown away. Bullets 1 and 2 above still apply.

Test-Driven Uncommenting

This is something I like a lot, because it is like a gentle version of throw-away prototyping. I need to work on the name a bit, but the general idea is:

1. Build the prototype. Use a high-level blackbox test to guide development.
2. Comment out all prototype code.
3. Use TDD to demonstrate the need for the commented-out code, uncommenting as you go.

The important thing here is that the TDD activity can be carried out by the book. Normally, using Test-After Development is very dangerous since the resulting tests are very likely (at least in my experience) to be buggy (not detect regressions) and/or to test the wrong things. With Test-Driven Uncommenting (or TDU), you get the correct red-green-refactor cycle:

1. Write a test (always start with a base case) that initially fails.
2. Uncomment the smallest amount of code necessary to make the test pass, preferrably a single line. Thus, the test must prove that the code is necessary. If it cannot prove it, then the code might be a candidate for removal.
3. Refactor the uncommented code as necessary.
4. Go back to step 1.

When I do this, I usually discover two things. First, that the prototype code contains bugs—sometimes subtle, sometimes obvious. Second, that it is convoluted and contains unnecessary/redundant code. The end result is typically something that vaguely resembles the prototype but a lot slimmer and of course very much more stable.

A crucial mindset to have when writing tests in this fashion is that the tests should drive the design and implementation of the production code in the correct direction, as opposed to just parroting the design and implementation of the prototype. The prototype code is most likely a bit unclean and rough around the edges, and we definitely don’t want to write tests that reinforce that.

Put differently, pretend that the prototype code doesn’t exist (as if it was thrown away) when you write tests, but uncomment relevant and reusable parts of it for the actual implementation.

It follows that TDU is most useful when the structure of the prototype is decent. Otherwise, it might not make sense to uncomment code into a design that is wildly different than the one for which the code was written. On the other hand, I often find that there are pieces (methods, typically) worth saving in any case. YMMV!

As noted before, since the prototype has been created, the ground is now familiar and possibilities and constraints are known. It becomes a lot easier to have a good idea about the kinds of tests that need to be written on different levels (layers) in the code.

Summary

TDD and prototyping are excellent companions!

A prototype that is thrown away removes a lot of unknowns that otherwise would make TDD difficult, but doesn’t leak into the production code.

A prototype can be test-driven with a high-level blackbox test which later on is thrown away. Such a test is most likely an integration test of some sort, perhaps even a UI test. It is not a unit test, as unit tests tend to be tied to a particular implementation.

Using Test-Driven Uncommenting, you can prototype freely and then use proper TDD to bring in relevant code from the prototype in order to formalize it. The best of two worlds!

Please let me known your thoughts on this in the comments below!

---

1: With run-click-crash, you have to run the application, click buttons/links until you get to where the feature resides, then watch as the application crashes because you forgot some simple detail.