persistence stories arrive late

In one of last week’s architecture workshops (see databases as life-support for domain objects) our group briefly considered how the new product line might emerge gradually from a series of user stories. The Product Owner agreed that stories about persistence would probably be placed relatively far down the backlog, behind the stories that deliver the first several tranches of usable business value. The Product Owner wants to see and understand the business logic first, and only later will he care about “non-functionals” such as persistence and levels of service.

So if we truly design and build our application story by story, without being tempted to predict what the Product Owner will want in the future, then the database will not exist until after many of the domain objects. This is business-level support for @jbrains‘s assertion that the database should be developed last. And it knocks sideways the idea that the ‘architecture’ should be designed at the start of a project.

Update, 9 Nov 2016

I’d like to thank Joe Rainsberger for taking the time and trouble to resurrect his article, which had disappeared from the interwebs sometime during the 11 years since I originally wrote this post.

time-boxes focus the mind

Two events this week convinced me of the power of timeboxing. In both cases an activity was in danger of drifting, such that the effort expended would far outweigh the value (if any) returned. And in both cases we applied timeboxes that seemed crazy at first glance, but which created the focus to get the jobs done.

In the first case, a real-live paying customer reported a bug. The current release plan says we’ll continue with 2-week iterations until mid-June, at which time the new release will be made available to all comers. But it was important to get an interim release to this particular customer much sooner than that. In fact, as soon as possible. Yesterday.

Now this team are currently making great strides towards agility, but their product and their code still show strong traces of the past. So the first estimate to produce the bug-fix release was that it would take a couple of hours to fix, and then tradition requires the entire team to test any release for 5 days! This estimate seriously crippled the release plan, pushing important stories out past the projected June release date. This, in turn, created an “atmosphere”, in which everyone looked for places to dump the blame. A project that was riding high was suddenly brought crashing to the ground by one defect that had been inserted into the code long ago.
Continue reading

energising the iteration review

Today was the last day of the iteration, and so I decided to make some changes for the iteration review meeting. One month ago the day had gone very badly (see the product owner must pull), so today we needed to recover a great deal of lost trust, credibility and morale. Here’s what we changed:

we moved out of the boardroom
Previous iteration reviews had been held in the boardroom. They were formal affairs, in which the development teams sometimes seemed to be almost on trial. The imposing setting made the directors – who are the Product Owners for the company’s various products – feel at home and in control. But it made the developers feel as if they had been called into the headmaster’s office. So this time we held the day in the team room. In the engine room. Busy desks, PCs set up for pair programming, information radiators all around the walls, and all dominated by the task board at one end. A place of work, and a change of dynamic.
we didn’t prepare a demo
The tradition had been to prepare a releasable CD and install it on a clean machine in the boardroom; this installation would then be used to demonstrate the increment’s features. We decided that the effort required to prepare all that was greater than the benefit we would gain by “practising” a release and installation. So today each new feature was demonstrated on a developer’s PC. We walked the Product Owner through the completed stories column on the task board, and for each story we turned to the developer who knew the most about that story. The developer showed the feature on his own PC, with the audience crowded around him. No PowerPoints, no clean install. But loads of energy. There was a real buzz as each story card in turn was demonstrated – a real connection between the backlog and the working software.
we held an iteration retrospective
After the increment demonstration we held a time-boxed iteration retrospective. A lot had changed since the last iteration boundary, and the team gained much by reviewing the progress it had made. It turned out we had recorded a roughly 20% increase in velocity – and a 100% increase in morale. The whole team committed to measure likely areas of muda in future iterations, and individuals committed to improving specific aspects of their own processes. By the end of the time-box the team really felt like a team.
we had a full backlog
After the previous iteration review we had implemented iterative backlog creation. So after the retrospective our analyst was able to present loads of stories for consideration at the top of the backlog. We put them on cards on the task board, and the Product Owners moved them around until they had selected one iteration’s worth of business value for us to develop next. Instead of PowerPoints in the boardroom, we had the company’s directors physically engaged in the project. The team watched with broad grins as the cards were touched, discussed, moved, pinned, unpinned. At one point we had two directors squatting on the floor looking at the rest of the backlog for comparison. Suddenly everyone was connected.

The day was a complete turnaround from the previous iteration boundary, and the team is now fired up and raring to go…

the product owner must pull (revisited)

In the product owner must pull I described what can happen when there are not enough stories in the pile. Here I want to look at the same problem from a slightly different angle: in terms of the Theory of Constraints.

The development team’s throughput is low. TOC says that the organisation must therefore include a Constraint (a bottleneck). The Constraint is the major factor in causing that poor productivity, and our first step is to IDENTIFY it. In software development, one of the easiest ways to find bottlenecks is to look for stuff piling up. In this case we find that the constraint lies outside the team, in the person/process/team that creates the story pile. There are piles of visions, wishes and capabilities waiting to be analysed, but downstream there’s no work for the developers to do. The constraint is the analysis function.

Our second step must be to EXPLOIT the constraint – to get it functioning on full power. In iterative backlog management I adopted Scrum to help the analysts generate stories and incrementally build a pile of stories for the developers to work on.

Third, we must SUBORDINATE everything else to the constraint. In this case we decided to stop development for one iteration. In lean manufacturing terms the development has no customer pulling features. So instead the development team will carry out some maintenance work: refactoring the codebase (tidying gemba), improving test coverage, installing new tools etc. (As things turned out, analysis generated some high-value stories quite quickly, and so we were able to re-start development on those stories before the end of the current iteration.)

Fourth, we must ELEVATE the constraint. Which means we must look for ways to make the current constraint more productive, or else eliminate it from the production flow. In our example this probably won’t be an issue. As the analysis team settles into being proactively managed it looks to be capable of creating stories faster than they can be developed.

Which leads us to the final step: we must PREVENT INERTIA and re-start the process with step one. Now that analysis is creating a story pile and development is working on it, we need to IDENTIFY the system’s constraint all over again. This time it will be somewhere different, and again we’ll find it by looking where things seem to be piling up…

iterative backlog management

When the Product Owner doesn’t have enough stories to drive development, try running an incremental analysis project that keeps one iteration ahead of the developers. The deliverable from each analysis iteration is a modified Product Backlog, ready for the upcoming development iteration’s planning meeting.

Whatever the reasons, it’s sometimes a fact that the Product Owner just doesn’t have the backlog prepared in time for the next development iteration. This can have a destructive effect on the development team’s morale and productivity, and must not be allowed to continue for too long. So something we’re trying on a current project of mine is to set up a parallel “analysis” project, to be run using the Scrum process. (This is certainly not a new idea; for example, Martin Fowler talks about this technique in Planning and Running an XP Iteration.)

I expect the mechanics of the two projects to work something like this:
Continue reading

use spikes to manage technical risk

At the AgileNorth meeting on Monday of this week we recapped an old discussion about technical risk. Some months ago Phran Ryder had raised the question of how to prioritise stories during release planning. I have always held out for business value as the only factor to consider. Phran however insisted that his project was different, and that technical risk was more important. The topic came up again this week, and this time Phran was able to provide a very succinct explication of how we resolved the issue.

The dilemma is this: In order to deliver maximum value under all possible outcomes, we should prioritise our stories solely on the basis of their value as perceived by the Customer. But what if we can’t assess that value, or estimate the stories, in advance? What if we need to bring some stories forward to the front of the schedule, so that by doing them we gain further information to help us create the remainder of the plan?

Here’s a quick attempt at a conflict cloud:

spikes

(I’m not too pleased with this – please help if you can think of a better formulation of the dilemma.)

We resolved the dilemma by attacking the necessary condition at bottom-right in the diagram: we reminded ourselves that implementing a story isn’t the best way to obtain information about when to implement it! That’s the job of a spike (a throw-away prototype). We should do just enough work to obtain the information we need about the story. Just enough to put us in a position to correctly prioritise or estimate it and get on with planning the release. (At this point we must also throw away the spike code. Because the odds are very good that the world will be a different place when the spiked story comes to the top of the pile. If we kept the spike, it would likely require learning and re-work to fit into the new and growing system at that time. Best to just ditch it now and start afresh when the customer really needs it.)