silk and spinach

A couple of weeks ago Clarke Ching interviewed me as part of his Everyday Agile initiative. The discussion describes what happened when one of my clients, Codeweavers, invited me to participate in kicking off a new project. You can download the MP3 directly from here.

Warning: The sound quality and levels are very poor, because my headset had packed up. So we’ll probably be re-recording it (or something similar) soon.

In the meantime I’m interested to hear from you if you’ve done the same kind of exercise, and how it turned out.

Clarke asked me to explain my earlier throw-away remark that YAGNI forms part of the EXPLOIT step in moving the bottleneck away from development, so here goes…

YAGNI (You Aren’t Gonna Need It) is an exhortation from the early days of XP. It has been discussed and misunderstood a great deal, so I’m not going to get into the finesses of meaning here. For our purposes, it reminds the developer not to work on features or generalisations that may be needed, telling him instead to focus his present efforts on delivering only what he knows is the current requirement. (In the interests of brevity, I’ll refer below to YAGNI only in terms of added behaviour, and I’ll use the word “feature” for any fragment of any kind of behaviour; all other forms of YAGNI are assumed.)

(In my practice I use a similarly attention-grabbing soundbite. Whenever I see a developer do something “because it may be needed in the future” I accuse him of crystal ball gazing. I remind the whole team that it can be risky and dangerous to get your balls out, and that seems to help the message stick. Other times there’s an embarrassed silence.)

Writing crystal ball code has three effects: In the present moment, it means that the developer is spending current time investing in one of many possible futures; in the period from now until that possible future, it means that there is code in the system that doesn’t need to be there; and when the future arrives, it may look different than that which the developer predicted.

First, then, crystal ball code uses up current development time. This is bad when development is the bottleneck and when batch sizes are relatively small and when development order has been defined in terms of business value and when feature cycle time is a KPI. The time spent developing a crystal ball feature will delay the current batch and all batches upto the imagined future. There is a tiny chance that development of that future batch will be faster (see below), but all interim ROI (for example) will be reduced by the delay introduced right now.

Second, the crystal ball code represents inventory, and it has a carrying cost. This code, which may never be required by the end user, must always build, integrate and pass all tests; if ever it doesn’t, time must be spent fixing it. Furthermore, a larger codebase will always require more time and effort to understand and navigate (think of having to drive around piles of inventory in order to fetch anything or the lean practice of 5S). Even if the guess turns out to be correct, the additional carrying cost of this inventory will slow down the development of all batches of features between now and the imagined future.

Third, the developer’s guess may be just plain wrong. Either the imagined “requirement” is never requested, or it is requested and by that time the codebase is radically different from what it is now. The developer may have to spend time removing the feature (for instance if it would confuse or endanger the user) or completely re-design it to make it match how reality turned out. It is assumed that the “wow, that’s exactly what we needed” outcome is sufficiently unlikely that the costs of the other outcomes dominate.

So YAGNI is based on a few core assumptions:

• The product is to be built incrementally in batches of features
• Each increment should be potentially shippable in terms of quality and cohesiveness
• It is hard to predict what features will be requested in later batches
• It is hard to predict what future code may look like
• Development is the bottleneck
• Speed of development is crucial
• The present value of current features is higher than the future value of future features

Under these conditions, YAGNI is part of the EXPLOIT step because it helps to maximise the amount of current development effort going into delivering current value.

My apologies if this has been said or written a thousand times before: YAGNI is XP’s way of exploiting the constraint.

Which means that XP, and hence most agile methods, are set up on the assumption that the development team is - or soon will be - the bottleneck. And having identified that bottleneck, our first task is to EXPLOIT it - that is, we make sure that the bottleneck resource only works on stuff that contributes to overall throughput. YAGNI and test-driven development do that. Oh, and a relentless pursuit of quality, so that the bottleneck doesn’t have to spend time on rework. And effective communication, so we get to spend more time developing and less time writing documents and attending meetings. And tight feedback loops, so that we can identify which stuff is valuable and which isn’t.

Next we must SUBORDINATE the whole flow to the pace of the bottleneck. Fixed-length iterations help us to measure that pace, and the various forms of planning game and iteration commitment help to prevent work arriving too fast.

And only when all of that is working well is it safe to ELEVATE the constraint, perhaps by expanding the size of the team. I’m fairly sure I’ve never seen a real-life case in which this step was required. For most of my engagements, successfully exploiting the bottleneck caused the constraint to move elsewhere; and in the rest, the SUBORDINATE step revealed a deeper constraint elsewhere in the organisation.

The Extreme Programming page on uncyclopaedia made me laugh out loud. (Particularly the laptops in the photos.) Sad how so much of it rings true though…

(Thanks to John for sharing the link.)

In Hitting the High Notes Joel Spolsky makes a compelling argument in favour of hiring only the best programmers. I agree with his conclusion, if not with his supporting arguments.

Joel presents tables of data showing huge variation in the times taken by many students on a series of programming exercises. He then goes on to imply that this means there’s huge variation in these programmer’s talents in general, and that therefore one should seek to hire only the “best” if one wishes to be successful. My problem is that I find that to be a very narrow - and possibly short-sighted - definition of “best”. So one programmer can pass the professor’s unit tests four or six times more quickly than another. Does that mean the faster feller will be a constructive team member? Or a good designer? Or that his code is more adaptable in the long term? I do agree that there is huge variation among developers. And I do agree that a successful team must comprise “the best” almost exclusively. But a good team must also be a good mix, and “the best” must therefore be defined more broadly than in terms only of programming speed.

As a case in point, I’ve long been concerned about the “first” XP project, C3 at Chrysler. To many, the project’s success was a result of the strict adherence to being “extreme”. But I suspect that any project would be likely to succeed when the staff include Kent Beck, Ron Jeffries and Martin Fowler. The AgileNorth group have discussed this question many times without a conclusion: does the success of XP depend on the presence of highly gifted individuals, or can a team of average developers make it work? I believe the former to be true, and it may be true in general of any software project, XP or otherwise. Which I guess is the point that Joel is also making.

We welcomed a couple of new faces to last night’s meeting of the AgileNorth group. Interestingly, both Josh and David want to get their work environments moving over to being agile, without either having tried it for real. So we spent a very pleasant evening out in the sun talking about how to sell agile into distinctly non-agile organisations.

One thing that struck me as fascinating: there seems to be a grand misconception that “being agile” is about TDD and pair programming. It isn’t - there’s far more to being agile than choosing the right programming techniques. Perhaps Kent Beck’s success in promoting XP has obscured part of the picture?

My own recent experience suggests that “XP” or “Scrum” or even “agile” isn’t the issue. As we discussed last evening, selling agile is really about delivering ROI quickly, incrementally and predictably. The techniques of XP will definitely help us turn the dials up to ten, but that’s rarely needed in the immediate term…

The AgileNorth group helped run an XP Game last night in Manchester. The event was organised jointly by ourselves and the good folks at Exoftware, on behalf of the Agile Alliance Europe programme. There were twenty-nine of us, including our facilitator David Putnam of Exoftware, and we all had great fun. We worked in 4 groups, coached by me, Brian Hanly of Exoftware, Murray Tait of Deltapoint and Stephen Hutchinson of Accenture. My group finished last!

Now, if I can find a couple of coaches, I’d love to try to run the game at my current place of work. Might as well be hung for a sheep as for a lamb…

I’m really excited: I’ve heard a great deal of praise for the XP Game, and now our local agile group is going to run one! It’s in Manchester on October 13th - see http://www.agileallianceeurope.org/uk_home/ManInvite for details.

The more I think about it, the more convinced I become that selling agile at all is a mistake. Certainly as a coach/manager, what interests me most is the ‘process’ side of things, and agile is always an interesting and rewarding approach for both me and (I hope) the software team. But what interests my bosses is more often the business value delivered to the user base.

I’ve never been someone who believed that it works to simply plonk a methodology onto a team straight from a book. Every team is different, and every business is different; enforcing conformance to some pre-determined monolith is likely to bring out as many minuses as plusses. And I believe that’s true of the agile methods just as much as it is of the heavyweight ones. What’s more, the successes I’ve been associated with have all been due entirely to having the team grow their own processes - gradually, incrementally making the changes required by the current situation. Only when the current way of working is shown to fall short in some respect does the team call a review and work out a different approach. There is never any change simply for the sake of it.

Of course the agile movement directs and informs my thinking, and hence my coaching and management styles. And all of the teams I’ve coached have become increasingly agile over time, without any of them ever consciously adopting a ‘book’ method. The stakeholders get to learn that iterative and incremental delivery is better for everyone, that feedback loops must be short, that quality helps the team go faster, etc etc. But most will never hear the words ‘agile’, ‘SCRUM’ or ‘extreme’.

In the future though, I do expect I’ll be using the word ‘lean’ a lot…

There’s been a thread recently on Clarke Ching’s SellingAgile list about selling agile to developers. My approach has always been to ‘just do it’ and lead by example. Here’s the story I told on the list:

I too was faced with this problem about four years ago. I believed the book, but I had never tried it for real, so why would anyone else believe me? So I just started to follow the advice in the XP white book on my ‘own’ subsystem (yes, it was that kind of shop). After a couple of months I started to get the hang of things, and ‘my’ software started to be much better. At that point other developers began to notice things like the hourly build and the test suite (and the lack of bugs ;-). Soon most of them wanted to try it, and we were off down the agile road.

(As it happens, potential management objections never materialised, because I was soon promoted to lead that project. I’d like to think that was because of my natural charm and leadership abilities, but more likely it had something to do with working software delivered on time and with fewer defects…)