a ruby sparkline showing variation

Back in May’s Carnival of the Agilists I referenced a post by Clarke Ching in which he suggests we can learn a lot about variation in a complex process by simply flipping coins. When I tried the simulation a few times with Excel I found, as expected, that heads and tails don’t always occur in equal measure. But that was a pain to do, so I’ve made it easier. I’ve written a little Ruby program that simulates 1000 coin tosses; you can get it from github.

The simulation generates SVG images; if you can view those in your browser, you can see three consecutive runs here, here and here.

(Disclaimer: The images are SVG. It works fine in Firefox, but if you use IE you may need to install some kind of viewer.)

developing a sense of urgency (revisited)

I posted a copy of developing a sense of urgency to the scrumdevelopment group on Yahoo!, and it sparked off a really great discussion. Among the wise words, Laurent Bossavit posted this:

I think the important thing you brought to the table with 2-point estimates is that an estimate is just what the probability density graph implies – it’s not one number, but a distribution.

What the team might have forgotten is that if you consistently give out conservative estimates, then 90% of the tasks should be finished way early, and therefore being mostly “on time” is very poor performance.

Much better to have a system where being “on time” is a challenging but achievable objective. No blame accrues for being “late”, and some praise is due for being merely “on time”.

Earlier, Simon Baker asked: “Could you elaborate on why you think the new estimates gave your team a sense of urgency?” To which I replied:

I suppose part of the answer lies in the fact that I didn’t tell them about the buffer ;-) Seriously. I only talked about the “luck” thing: With conservative estimates and Parkinson, we never benefit from good luck (ie. when something is easier than we had imagined it would be). But we’re always hit by the slightest hint of bad luck – and in an iteration of 20 tasks, one of them is dead certain to go badly. The team had complained – at several successive iteration retrospectives – that they got squeezed at the end of the iteration, no matter what they tried. So I talked about luck. No mention of buffers. No graphs or fancy science (actually, this was the hardest part for me, a Maths PhD ;-).

Another big part of the answer is that, once we got into the iteration, they saw they were finishing tasks. The team room started buzzing a couple of days into the first iteration we tried this, because tasks were moving rapidly into the ‘Completed’ column on the board. And as more cards moved, the team realised that they were in a space they’d never been to before, and the buzz increased. I guess the Big Visible Chart showing completed tasks acted as a positive feedback loop.

As I say, the whole thread makes very interesting reading…

developing a sense of urgency

In Estimating Project Durations – not Deadlines, Frank Patrick rails against the demand for “accurate” estimates and the ensuing deadline culture. His very good article reminds me of something I’ve been meaning to post for a while, so here goes.

On this particular project, just over a month ago, I decided to try an experiment. Our process involves taking user stories and breaking them down into small development tasks which, being small, should be relatively easy to estimate. But iteration after iteration we were squeezed at the finish post because some tasks had taken way too long. I had no idea what the cause might be, but in my mind I coupled it with the team’s overall apparent lack of urgency. Things just drifted along.

So just before the next iteration’s planning meeting we had a team workshop. I explained the critical chain view of estimates, and we decided to give it a go. The team recognised that historically all of its task estimates had been conservative. Fear of being late had made us create estimates that were universally over at the 90-100% end of the range of possibilities for each task. That is, each estimate created a 90-100% chance of “being right”, of finishing within the estimated time. So we had stacked each iteration with 15-20 small tasks, each of which had only a small chance of being late. And each time, one or two were indeed late. No-one knew why, and yet those few late tasks ate up the iteration’s slack. Every time.

So we removed the pressure on any individual task to be brought in “on time”, and we switched to estimating every task at the 50% point. That is, we did our usual conservative estimates and then halved them! The results were amazing, and have been repeated now over three iterations:

Most tasks are now finished by the 50% point (partly because the developers treat the estimate as a timebox – “I’ll do what I can in that amount of time, and then see where we are.”) And those that go over just eat into the project buffer a little. In fact, we’re now getting a whole load more tasks done in the same time. Critical chain estimating has eliminated the effect of Parkinson’s Law and simultaneously created a sense of urgency.

I’d love to know whether this works for other software teams. So if your iterations consist of a suitably large number of tasks, and if your estimating is generally conservative (read “fearful”), just try halving every estimate and see what happens.

This article provoked a great deal of discussion on the Yahoo scrumdevelopment list – read a summary in my follow-up post.

critical chain

On Clarke‘s recommendation I just finished reading Critical Chain by Eli Goldratt. This is another of his “novels”, in which this time he applies the ideas of TOC to project management. I liked the book’s use of the TOC thinking tools. And I also liked the discussion in the last couple of pages, about using dollar-days as the unit to measure investment – although I need to think about it a lot more before I’ll come close to understanding it.

The main thrusts of the book are that managing slack at the project level is better than managing it per task, and that the critical path/chain is the bottleneck in a project, and therefore needs to be protected. Again, I need to think about this a lot before I can fully see how – or whether – these ideas will apply to an agile software project…