keeping innovation alive

In Keeping Innovation Alive – The Hackathon Joe Kraus describes a one-day event at his company, in which the developers all get to work on something new and different. Normal development stops for the day, and everyone has just that one day to produce something, anything they want to. The idea was also tried over at Feedburner. Both blogs provide details of their rules for the day, together with a full round-up of the results they achieved. Which are pretty impressive.

I’m wondering about two variants on this idea. In the first, the day is spent fixing those little niggly things that get up someone’s nose. We could begin with an open session in which everyone – including sales, marketing, directors – whiteboards their pet peeves. Then developers just sign up for anything that takes their fancy and spends the rest of the day fixing it.

In the second variant, it’s not just about development. Perhaps all of the other departments in the company could stop for the day too, and fix pet peeves in their own bailiwick. Or maybe we could get cross-department pairs going on the fixes…

developing a sense of urgency

In Estimating Project Durations – not Deadlines, Frank Patrick rails against the demand for “accurate” estimates and the ensuing deadline culture. His very good article reminds me of something I’ve been meaning to post for a while, so here goes.

On this particular project, just over a month ago, I decided to try an experiment. Our process involves taking user stories and breaking them down into small development tasks which, being small, should be relatively easy to estimate. But iteration after iteration we were squeezed at the finish post because some tasks had taken way too long. I had no idea what the cause might be, but in my mind I coupled it with the team’s overall apparent lack of urgency. Things just drifted along.

So just before the next iteration’s planning meeting we had a team workshop. I explained the critical chain view of estimates, and we decided to give it a go. The team recognised that historically all of its task estimates had been conservative. Fear of being late had made us create estimates that were universally over at the 90-100% end of the range of possibilities for each task. That is, each estimate created a 90-100% chance of “being right”, of finishing within the estimated time. So we had stacked each iteration with 15-20 small tasks, each of which had only a small chance of being late. And each time, one or two were indeed late. No-one knew why, and yet those few late tasks ate up the iteration’s slack. Every time.

So we removed the pressure on any individual task to be brought in “on time”, and we switched to estimating every task at the 50% point. That is, we did our usual conservative estimates and then halved them! The results were amazing, and have been repeated now over three iterations:

Most tasks are now finished by the 50% point (partly because the developers treat the estimate as a timebox – “I’ll do what I can in that amount of time, and then see where we are.”) And those that go over just eat into the project buffer a little. In fact, we’re now getting a whole load more tasks done in the same time. Critical chain estimating has eliminated the effect of Parkinson’s Law and simultaneously created a sense of urgency.

I’d love to know whether this works for other software teams. So if your iterations consist of a suitably large number of tasks, and if your estimating is generally conservative (read “fearful”), just try halving every estimate and see what happens.

This article provoked a great deal of discussion on the Yahoo scrumdevelopment list – read a summary in my follow-up post.

15% time

Value flows quickly through a lean factory all of the time. This high throughput is sustained because faults in the product and process are continuously removed. But in order to achieve such continuous process maintenance, the plant must be over-staffed. A lean factory employs more than the bare minimum number of people required to support current flow rates. Because the preservation of tomorrow’s flow rates depends on eliminating systematic problems. And in a constantly changing world that requires effort.

I read somewhere that Toyota over-staffs by about 15%. That is, workers spend about 85% of their time adding value, and the other 15% ensuring that the current pace is sustainable. Note that value is being added 100% of the time, and product is flowing at the rate demanded by the market. It’s just that by adding one extra person in every six, there’s now a little slack in each team. So when Murphy strikes, the team’s process can be fixed without diverting effort from adding value.

Some Western organisations have taken this 15% slack time and cordoned it off. “Everyone spend 15% of your time improving our processes,” they say. This is undirected effort. Better to have that slack available, and to fix real problems as they occur. This contributes more directly to protecting the value stream.

So an agile software development team should only commit to delivering 85% of what it knows is possible each iteration. The remaining 15% should be used for kaizen activities, and for dealing with quality incidents as they occur. Without that slack, bad luck will either impact today’s schedule or tomorrow’s quality. Either way throughput will be compromised, and the project is on the road to failure.

adding value

I once worked alongside a chap who was a project leader in a large organisation. His team produced a new release of their system every 4-6 months, and so over a few years he ran quite a sequence of projects, all on the same codebase and all with the same team. He’s a bright guy, good at his job, one of the nicest people I ever worked with, and his story is quite typical. I’m sure you’ve seen something similar to what I’m about to describe.

During the second and third of these projects he diligently noticed that his programmers were spending a portion of their time fixing defects in previous releases. He diligently measured the time thus spent – at 10% of the project’s staff-time. And this was his perfectly rational reason for missing his deadline and dropping a couple of features into the bargain. During ‘planning week’ for the next round of projects he therefore proposed to set aside 10% of his staff’s time for fixing defects. He had the figures to prove that this was sensible, and the programme managers supported his plan. (In fact, they encouraged all of his peers to do the same.)

So for the next release his team spent 10% less time adding new features to the system. A thinking person might then have expected that the following project’s time budget would thus need to be shortened by only 9% – because surely 10% less product will have 10% fewer bugs, right? Wrong. The 10% figure never tailed off, and no-one ever noticed.

So either the bug-fixing time was being used to mop up other problems in the estimates, or the bugfixes contained as many bugs as the original code. At one point I suggested he spend the 10% of all future projects in implementing defect-prevention measures, but he didn’t have the time…

critical chain

On Clarke‘s recommendation I just finished reading Critical Chain by Eli Goldratt. This is another of his “novels”, in which this time he applies the ideas of TOC to project management. I liked the book’s use of the TOC thinking tools. And I also liked the discussion in the last couple of pages, about using dollar-days as the unit to measure investment – although I need to think about it a lot more before I’ll come close to understanding it.

The main thrusts of the book are that managing slack at the project level is better than managing it per task, and that the critical path/chain is the bottleneck in a project, and therefore needs to be protected. Again, I need to think about this a lot before I can fully see how – or whether – these ideas will apply to an agile software project…