A few days ago I noted that Shigeo Shingo, one of the founders of lean manufacturing, once said that testing to find problems is waste, whereas testing to prevent problems is not. Today I’ve been helping out with configuring an instance of CruiseControl, one that runs three or four projects and checks that no-one has broken any of them. This is testing after the fact, after check-in, after the developer has written his unit tests and his code. So is it waste?
I think it’s quite a close call, but on balance I’ll differ with James Shore and answer “no”. There’s no getting away from the fact that CruiseControl is reviewing code after the fact. After commit. We have to break our project before CruiseControl helps us out. And I wonder whether the presence of a good, automated tester downstream can sometimes engender a little too much comfort. I know I’ve been guilty of saying “Oh let’s just check this in — CruiseControl will soon tell us if we’ve made a mistake.”
But in CruiseControl’s favour, it certainly does catch the most frequent problem I’ve seen with (developers’ use of) version control: forgetting to commit a new source file. And by running a complete, clean build, it can throw up environmental or portability problems not seen when developers work for long periods in stable, cosy workstation setups. So it’s a fresh pair of eyes, doing a quick review of the project’s latest state. Possibly akin to automated checks of a sub-assembly in a lean manufacturing plant, it catches some problems early in the flow.
Maybe in the end it’s about the team. Installing CruiseControl is a great first step on the road to being “always release ready”, and definitely helps teams transition to agile development. And then after a while, I wonder whether there’s a level of team maturity at which it may no longer be needed?