We had the good fortune of starting Guidewire at a transformative time in the world of software engineering. It was in the early 2000's that people started really rethinking what it meant to write testable software. Some of the progress came through the advent of new tools and language constructs and from the sheer availability of computing power to do meaningful large-scale automated testing. But the real difference, for us, was in switching our heads around to put testing first.
It turns out writing those tests first has a bunch of interesting consequences. Here are a few:
We get lots of unit tests that cover a lot of our code. In general, having more tests is a good thing. (When it is not a good thing is also interesting, but I'll save that for another entry.)
Because the tests are written by the developer who is about to write the implementation, they are blessed with a high degree of connectedness to the original goal of the developer. Contrast that to a set of code that is shipped (across a table, room, building, or a few continents) to a different place and placed in front of a different brain. Or contrast it to the code being in front of the same brain, but later -- when that brain has perhaps had some other distracting adventures in the meantime.
Thinking of how to express what the code should do, in code, is a refreshing way to visit a problem. It takes time for a developer to learn how to think in "test-first," but figuring out how to express the requirements in a programmatic way often helps point to a particular design -- or it might rule out one that would have been tough to test. Which leads me to the next consequence:
Writing tests first produces implementation code that is more testable! Since we spend our time writing code that has to serve our customers for decades, we want that code to be supple and flexible with respect to testing, both today and tomorrow. Maybe later we'll find a bug, and we'll want to expand the testing in an area. Or, maybe new test capabilities come available in the future and we want to be able to leverage them against what we've already built. Code that's designed for testability is just different.
We do get high quality software deliverables by taking this aggressive approach to test engineering. We have hundreds of thousands of unit tests in our codeline today, and we run millions of tests every day. Our developers, testers, writers, and product manages can take builds early in a release cycle and have software that already works. When we do cut a build for customers, even the ".0.0" release will have very high quality. That's all well and good, and I could stop there. But there is one more benefit from having broad, fine-grained tests: confidence!
In my last post, I talked about needing to be able to curate the platform over time. Many times, we face the challenge of replacing an older component with a new one. We have to do this to a codebase that supports core systems running in hundreds of environments around the world. In the most complex cases, it can be a little bit like a heart or lung transplant. And this is where that giant library of tests really comes into its own: we can swap in a large new block of functionality, push a button, and watch the rest of the system validate correctness against the new "heart!"
What we discovered is that a really rich test suite becomes an informal specification for the entire product. This kind of enforceable fidelity gives us the ability to innovate, change and evolve our code with high confidence that we can come out of the procedure with a healthy "patient," still working the way he should. We've done quite a few of these "procedures" over the last ten years, and I'm convinced that our investment in test is one of the most important things we can do to serve our customer family faithfully in the long term.