I just had a thought about the relationship between software development and the Theory of Constraints. It probably isn’t a new thought, although it seems to differ from some of the analyses I’ve seen elsewhere. Also, I probably won’t be able to express it in any coherent way; but here goes…
This is a quick note about an idea I’ve been using with a few software teams during the last couple of years. I also spoke about it briefly at the Scottish Ruby Conference this week (here’s a video of the conference lightning talks – mine begins at 07:54). If you try it, please publish your experiences and link to them via the comments here.
TL;DR — don’t guess the size of a story; fit the story to the size you want.
Last night was the October 2010 meeting of XP-Manchester, a local group set up by me and Jim McDonald. As always the meeting consisted of two halves, the first being a workshop (this time led by me) and the second being a coding dojo.
For the workshop this month I ran a version of James Shore’s Offing the Offsite Customer game, as described by Kane Mar and using Kane’s drawings as the requirements. I hadn’t run the session before, and it turned out really well. We had 19 participants, so we split into two roughly equal-sized teams, with each team further split equally between a group of Product Owners and a group of Developers.
In the first run-through neither team managed to create a diagram that looked anything like the requirement; whereas in the second attempt both teams produced very good diagrams, and well inside the allotted time. The difference was born in the team retrospectives between the two runs. Both teams independently decided to work much more iteratively and interactively second time around, and it paid off. It could be said that in the first run, the teams focussed on perfecting the written spec, whereas in the second run the teams focussed on perfecting the working diagram. This focus on evolving a diagram using direct feedback was “invented” independently by both teams, and towards the end of the second run they even had free time available for fine-grained polishing.
The second part of the evening was a dojo. Jim introduced us to the Minisculus challenge set by Eden Development at the recent Software Craftsmanship 2010 event. Jim decided we should attempt the Mark I problem in Ruby, and due to the relative lack of Ruby knowledge in the room last night this meant that Mike Josephson did most of the driving. We didn’t get very far, but we did have some very interesting discussions about TDD style: After you’ve faked a return value to get quickly to GREEN, what’s the best step to take next? Is it better to add another test in order to triangulate towards a more general solution, or is it better to treat the fake return value as duplication and fix that by moving specifics up into the test? We explored the latter approach last night, and no doubt we’ll continue the debate next month.
Many thanks to Jim and Mike for running things, and to everyone else for joining in!
XP-Manchester happens after work on the second Thursday of every month at Madlab in Manchester’s Northern Quarter. Everyone is welcome, and if you want to come along you can get details of upcoming meetings by joining the mailing list at http://groups.google.com/group/xp-manchester.
Ths ten-minute video chat between Corey Haines and J.B.Rainsberger introduced a nice simplification of eXtremeNormalForm.
In the discussion, JB hardens up the wishy-washy Communicates Intent value by noting that it’s just about “bad names”, and here’s why. In an OO language, “communicating intent” boils down to breaking the system into small pieces and giving them good names, names that resonate with the design and the domain. Small pieces are no good on their own, and good names for bad ideas will soon be weeded out. JB therefore claims that good design (he says good architecture) boils down to
- eliminate duplication
- eliminate bad names
(He takes passing all tests – ie. correctness – as a given, and he says that eliminating duplication and bad names also renders the software “small”. I disagree on that last point.) So, his assertion is equivalent to saying that every code smell is a symptom of either Duplication or Bad Names, which I find easier to explain than asking people to ensure that the code “communicates intent”.
A couple of weeks ago Clarke Ching interviewed me as part of his Everyday Agile initiative. The discussion describes what happened when one of my clients, Codeweavers, invited me to participate in kicking off a new project. You can download the MP3 directly from here.
Warning: The sound quality and levels are very poor, because my headset had packed up. So we’ll probably be re-recording it (or something similar) soon.
In the meantime I’m interested to hear from you if you’ve done the same kind of exercise, and how it turned out.
YAGNI (You Aren’t Gonna Need It) is an exhortation from the early days of XP. It has been discussed and misunderstood a great deal, so I’m not going to get into the finesses of meaning here. For our purposes, it reminds the developer not to work on features or generalisations that may be needed, telling him instead to focus his present efforts on delivering only what he knows is the current requirement. (In the interests of brevity, I’ll refer below to YAGNI only in terms of added behaviour, and I’ll use the word “feature” for any fragment of any kind of behaviour; all other forms of YAGNI are assumed.)
(In my practice I use a similarly attention-grabbing soundbite. Whenever I see a developer do something “because it may be needed in the future” I accuse him of crystal ball gazing. I remind the whole team that it can be risky and dangerous to get your balls out, and that seems to help the message stick. Other times there’s an embarrassed silence.)
Writing crystal ball code has three effects: In the present moment, it means that the developer is spending current time investing in one of many possible futures; in the period from now until that possible future, it means that there is code in the system that doesn’t need to be there; and when the future arrives, it may look different than that which the developer predicted.
First, then, crystal ball code uses up current development time. This is bad when development is the bottleneck and when batch sizes are relatively small and when development order has been defined in terms of business value and when feature cycle time is a KPI. The time spent developing a crystal ball feature will delay the current batch and all batches upto the imagined future. There is a tiny chance that development of that future batch will be faster (see below), but all interim ROI (for example) will be reduced by the delay introduced right now.
Second, the crystal ball code represents inventory, and it has a carrying cost. This code, which may never be required by the end user, must always build, integrate and pass all tests; if ever it doesn’t, time must be spent fixing it. Furthermore, a larger codebase will always require more time and effort to understand and navigate (think of having to drive around piles of inventory in order to fetch anything or the lean practice of 5S). Even if the guess turns out to be correct, the additional carrying cost of this inventory will slow down the development of all batches of features between now and the imagined future.
Third, the developer’s guess may be just plain wrong. Either the imagined “requirement” is never requested, or it is requested and by that time the codebase is radically different from what it is now. The developer may have to spend time removing the feature (for instance if it would confuse or endanger the user) or completely re-design it to make it match how reality turned out. It is assumed that the “wow, that’s exactly what we needed” outcome is sufficiently unlikely that the costs of the other outcomes dominate.
So YAGNI is based on a few core assumptions:
- The product is to be built incrementally in batches of features
- Each increment should be potentially shippable in terms of quality and cohesiveness
- It is hard to predict what features will be requested in later batches
- It is hard to predict what future code may look like
- Development is the bottleneck
- Speed of development is crucial
- The present value of current features is higher than the future value of future features
Under these conditions, YAGNI is part of the EXPLOIT step because it helps to maximise the amount of current development effort going into delivering current value.
My apologies if this has been said or written a thousand times before: YAGNI is XP’s way of exploiting the constraint.
Which means that XP, and hence most agile methods, are set up on the assumption that the development team is – or soon will be – the bottleneck. And having identified that bottleneck, our first task is to EXPLOIT it – that is, we make sure that the bottleneck resource only works on stuff that contributes to overall throughput. YAGNI and test-driven development do that. Oh, and a relentless pursuit of quality, so that the bottleneck doesn’t have to spend time on rework. And effective communication, so we get to spend more time developing and less time writing documents and attending meetings. And tight feedback loops, so that we can identify which stuff is valuable and which isn’t.
Next we must SUBORDINATE the whole flow to the pace of the bottleneck. Fixed-length iterations help us to measure that pace, and the various forms of planning game and iteration commitment help to prevent work arriving too fast.
And only when all of that is working well is it safe to ELEVATE the constraint, perhaps by expanding the size of the team. I’m fairly sure I’ve never seen a real-life case in which this step was required. For most of my engagements, successfully exploiting the bottleneck caused the constraint to move elsewhere; and in the rest, the SUBORDINATE step revealed a deeper constraint elsewhere in the organisation.