Inventory in software development

One of the central tenets of lean (and TOC, for that matter) is that inventory is not an asset, but is waste. In particular, carrying inventory incurs storage costs.

American Bulk Warehouse Shopping

David Carlton is reading up on lean manufacturing, particularly in relation to applying its techniques to software development. This week David asks a deceptively simple question about inventory:

“… it’s not clear to me exactly what lesson [the carrying cost of inventory] has for software development”

While neither David nor I would claim that software development is similar to manufacturing, it can be edifying to map the themes, principles and practices of the one onto the other. I think the answer to this particular question lies in looking at the various kinds of inventory we have in software development, and examining the carrying costs of each in turn. Off the top of my head:

Unreachable code:
Still gets built, thus contributing to slower build times, and hence to reduction of flow and feedback during coding. It may also need to be read (or worse – understood) whenever we’re debugging or designing an extension in that area. We have to keep it compiling successfully, which means it could actually prevent design changes and cause inertia. And if it has tests, we have to keep them building successfully and passing – another potential cause of reduced speed, frustration and ultimately design inertia. (All of this because our codebase is part of gemba.)
Unused features:
Same as unreachable code, plus: Features we ship get in the way of our users, which will either confuse them, slow them down, or otherwise generally leave them feeling slightly uneasy about our product. And presumably our testers still have to ensure that the features do work as intended, so there’s extra pointless work to do for each release. And if these features have bugs, we have to carry the support cost when a user stumbles upon one.
Items in the product backlog:
Have to be reviewed whenever stories are prioritised. And if we have too many, they may contribute to a sense that the product is a Death March.
Artefacts that don’t ship:
By which I mean documents, plans and models that must be kept up-to-date as the fuzzy future comes into focus, and as the product evolves. None of these artefacts is directly a part of the product, and yet we spend time and money fiddling with them (while Rome burns).

(And we haven’t even mentioned the time and money invested in creating these things in the first place!)

What other inventory and carrying costs does your project have?

organise around value streams

In Excellence Is Not Enough Bill Waddell tells the story of Rittal, a German company with a plant in the US. The company has executed a startling turnaround in three years by implementing lean thinking from top to bottom:

“What particularly impressed me is that they approached lean from the opposite direction than that taken by most companies. They started with their organizational structure and made value streams the formal way they work, instead of the old functional departments the rest of us cannot seem to get past. They scrapped the data systems, thinking that just talking to each other was better than using bad data.”

horsecar This fits very well with my experience of transitions to agile in the softare development domain. It’s one thing to have a fast, effective, high-quality development team; it’s something better entirely when that team sits within a lean organisation. When the bottleneck is outside of development, it often seems as if we have a sportscar being pulled along by a donkey. And the message from Rittal is clear: as long as we retain functional divisions, the organisation is unlikely to improve much. Having an agile software development department is good; but having a software development department that is an effective part of a lean organisation where the money is.

process improvement metrics – some questions

Software process improvement is all very well, but how do we know whether it is working?

The topic of measuring the effectiveness of process improvement activities has crossed my path several times in recent days. For example my current client and I are trying to define ways to measure the benefits to their organisation of retaining me. And then there was the designing metrics session at SPA2006; in their handout notes, Jason and Duncan suggest that process improvement be run as an agile project, with its own stories, velocity etc. Not to mention a conversation I had with Deb Hartmann on the same topic; seems she wants to measure – or at least chart – her coaching activities.

blindfold I’ve found almost nothing written about this topic, so at present I’m afraid I have only questions. First though, some basic assumptions. I assume that software process improvement in itself is pointless. As I said earlier this week, the term “process” doesn’t capture enough of the behaviour of a system to be useful. And of course “improvement” is a relative term. Improvement compared to what? And towards which goal(s)? According to Goldratt there’s only one Goal: maximising profits in both the present and the future. I will assume, then, that software development is always carried out with that goal in mind: the software either reduces our own organisation’s costs or increases our sales, or is designed to be sold to other organisations so that they can use it for one of those ends. Either way, the process of producing that software is only a fraction of the whole system, and may contribute a large or small amount to how well we’re working towards our organisation’s Goal. Furthermore, improving the software development department in isolation may prove beneficial initially, but could be creating a local optimum at the expense of another part of the system, or could simply not be worth doing, if other parts of the system are contributing more to poor overall performance.

All of that notwithstanding, there will be times when an organisation must turn its attention to improving the software development department. Software development will inevitably be the system’s primary bottleneck some of the time, and so its contribution to the goal will need to be addressed. But depending on context the “improvement” required may involve improving quality, reducing time to market, improving customer service, ensuring the right software is developed, training people, or any number of other things. That is, the day-to-day dynamics of the overall system’s push towards the Goal will create short- and medium-term demands on software development; and the response from software development must be measured in terms of those demands.

Henceforward, I’m going to use the woolly term effectiveness as a shorthand for the software department’s contribution to the overall system Goal. Improving the department’s effectiveness then equates to reducing costs and/or increasing sales in the overall system. And instead of talking about “software process improvement”, I’ll discuss “software department effectiveness” instead.

So, back to the top. How can we measure the performance of a software coach? As yet I have no answer; instead I’ll ask a few more related questions:

  1. If we hire a coach or process facilitator or scrum master, how will we know whether we’re getting value for money? Unless the coach doesn’t charge, the organisation’s Operating Expense has increased, so should we be looking for a corresponding increase in sales?
  2. Should a software coach charge a percentage of their client’s increase in profits? Now and in the future?
  3. How often is it possible to attribute improved profitability to the actions of a software coach?
  4. What if the service constitutes training – how can that be directly accounted to profitability?
  5. Given that increased profitability may be measuable only after a period of time has elapsed, how should the risk be apportioned (between client and coach) during that interval?
  6. Conversely, how risky is it to define local measures purely on the software department?
  7. Imagine an ineffective organisation whose biggest problem is not within the software development sub-system. But suppose nevertheless that the organisation blindly hires an agile coach to improve the software department’s effectiveness. Is it meaningful to talk about “improvements” when the real problem is elsewhere? Can the changes be measured in any meaningful way?
  8. Assume that one of the objectives of a coach is to transfer sufficient knowledge so that the client organisation can ultimately continue alone. How can we chart progress towards that goal?

I can find no answers in the lean manufacturing literature (assuming lean sensei charge for their services), or in the agile domain. I suspect that we as an industry just don’t know (yet). Deb suggests we may need a few OpenSpace discussions to work out the answers – or even to establish some direction – and I concur. If you know of any resources, or if you wish to participate in getting these discussions going, please drop me a line.

more lean books

Many thanks to everyone who emailed me with details of books I missed from my lean manufacturing reading list. To pick out just a couple:

“The best source on ‘lean’ (in the broad sense) for product / software development is Reinertsen’s Managing the Design Factory. It’s fantastic. Lot of Mary and Tom’s stuff is from it.” — Clarke Ching

Thinking Beyond Lean is about the product development model of car manufacturers and how Toyota completely overhauled their product development process, at a time when some of their competitors were starting to copy Toyota’s process. See here for a first stab at mapping the ideas in the book to agile software development.” — Pascal van Cauwenberghe

Any others you can recommend?

lean manufacturing reading list

David Carlton emailed me to ask:

“I’d been thinking that I should read more about lean manufacturing; what are your favorite books on the topic? And am I correct in assuming that the Poppendieck book is where to start for lean software development?”

Lean manufacturing is indeed a fascinating topic, particularly in the week that Ford in the US seems to have completely failed to understand it…

I guess I began by going to the horse’s mouth to read Taichi Ohno’s Toyota Production System. After that there’s a wealth of material out there. Many of the most approachable are those written by Westerners for Westerners: such as The Machine that Changed the World or Lean Thinking by Womack & Jones, or Liker’s The Toyota Way. Others are more specialised, such as Imai’s Gemba Kaizen.

Lean manufacturing concepts – such as pull, jidoka, gemba etc – can be applied to software development with positive effect (see Lean Software Development by the Poppendiecks). However, beware of drawing parallels between software development and manufacturing. In a software production business, the “manufacturing” step is largely about packaging and burning CDs etc, whereas software development itself is what folks in hard engineering call “Product Development” or “Engineering”. Toyota’s approach to product development is often termed Knowledge Management or the Learning Organisation, and their performance there is perhaps even more startling than it is in manufacturing. Yet there’s little approachable material available. I began with Kennedy’s Product Development for the Lean Enterprise.

There is a growing body of information available about knowledge management and learning organisations – I’m currently working my way through Takeuchi & Nonaka’s Hitotsubashi on Knowledge Management. However, with the exception of influences in the Poppendiecks’ book I know of very little material mapping these concepts directly onto software development.

(Inevitably my reading has been patchy and is woefully incomplete. So please let me know if you’ve read books I’ve missed here – particularly any that deal more explicitly with mapping software development to knowledge management and learning organisations.)

Corporate muda – a recap

In case you missed them, or can’t find stuff in the archives, here’s a recap on the posts I’ve written about waste in large organisations…

The muda of walking about
Co-location, co-location, co-location
Your eyes are too high, sir
A salutary tale of bureaucracy without lean thinking
More on multi-tasking
Multi-tasking delays everyone
Local optimisation
In which cost accounting at the department level leads to false positives in the search for productivity improvements
The taxi business
When per-project cost accounting dominates, the first project that needs an infrastructure feature pays for it; everyone else then gets a free ride. Local optimisation prevents progress
Tidy gemba
House-keeping is never a waste of time
I only want a pencil
In a dysfunctional organisation, even pencils can be exorbitantly expensive

culture and agile adoption

In Observations on Corporate Culture and Agile Methods Adoption/Adaptation Esther Derby looks at the potential for adoption of agile methods in a variety of organization types. I found it well worth a few minutes thinking about the organizations I’m involved with, and asking myself which of Esther’s cultural categories they fall into. Mostly it confirmed my TOC analysis of where the bottlenecks lie, so Esther’s analysis “feels right” to me. Well worth a look.

Update, 10-jan-06
Coincidentally, yesterday Kevin Meyer wrote about Looking Lean vs. Being Lean, in which he cites people and culture as the most important factors in the lean transition. According to Kevin, “real” lean transformation artists can take two to three years just getting the organizational culture set up and ripe for the changes. Which chimes in nicely with Esther’s analysis…

Your eyes are too high, sir

Applications for UK passports must be accompanied by two photographs of just the right size, shape, brightness, contrast etc. There’s a double-sided A4 sheet with the application form, giving a dozen examples of acceptable and unacceptable photos, so that you have every opportunity to supply them correctly first time. And as if that’s not helpful enough, our Post Office provides a “check and send” service (for the mere consideration of £7), whereby an expert will go through your form and check that your photos conform to the required standards. If you’ve messed up, there’s always a photo booth right there, so you can rectify any problems immediately. And hence the Passport Office will receive a greater number of correct applications, improving their turnaround time and everyone’s satisfaction. Until now.
Continue reading

5-whys versus current reality trees

A while back I tried running Toyota’s full 5-whys process on every problem that came up on a particular project. Each time through, we found that the fourth and fifth levels regularly came up with the same old reasons – to such an extent that the process became somewhat devalued for us. Don’t get me wrong, we did find valuable insights some of the time, and consequently we were able to permanently eliminate some classes of problem. But usually those insights and fixes came from our values (ie. automate everything, test everything etc), and not really from the process itself. I have only rarely used full-on 5-whys since, preferring an informal team workshop approach to fixing root causes.

But recently I’ve been looking at Current Reality Trees (CRTs) from the Theory of Constraints. The main difference, it seems to me, is that a CRT begins with around ten undesirable effects (UDEs), whereas 5-whys begins with only one. So in a CRT, the interplay between the numerous problems seems likely to get to the common core of the problem more quickly. And that core problem is instantly validated, because we’ve derived it in such a way that we know it is relevant to many problems.

If you’ve used both techniques, did you find one more successful than the other? Did one generate more convincing results? Or are they answering different questions, and should they therefore be used in different contexts?

15% time

Value flows quickly through a lean factory all of the time. This high throughput is sustained because faults in the product and process are continuously removed. But in order to achieve such continuous process maintenance, the plant must be over-staffed. A lean factory employs more than the bare minimum number of people required to support current flow rates. Because the preservation of tomorrow’s flow rates depends on eliminating systematic problems. And in a constantly changing world that requires effort.

I read somewhere that Toyota over-staffs by about 15%. That is, workers spend about 85% of their time adding value, and the other 15% ensuring that the current pace is sustainable. Note that value is being added 100% of the time, and product is flowing at the rate demanded by the market. It’s just that by adding one extra person in every six, there’s now a little slack in each team. So when Murphy strikes, the team’s process can be fixed without diverting effort from adding value.

Some Western organisations have taken this 15% slack time and cordoned it off. “Everyone spend 15% of your time improving our processes,” they say. This is undirected effort. Better to have that slack available, and to fix real problems as they occur. This contributes more directly to protecting the value stream.

So an agile software development team should only commit to delivering 85% of what it knows is possible each iteration. The remaining 15% should be used for kaizen activities, and for dealing with quality incidents as they occur. Without that slack, bad luck will either impact today’s schedule or tomorrow’s quality. Either way throughput will be compromised, and the project is on the road to failure.