Lean and Agile: Partners or Enemies?

An increasing number of commentators on both lean New Product Development (NPD) and agile software development have noted efforts to combine the two. A key reason for the interest in the combination is that lean has long been entrenched in the manufacturing process (e.g., via Kanban and ERP), while agile development has recently established itself as an effective software development strategy, and a major segment of NPD now involves software-only products, software-infused products, or software-plus-hardware solutions. A recent Aberdeen Group study notes that 66% of today’s NPD uses software to drive innovation, and the number is rising rapidly. Meanwhile, several efforts are being made to apply lean-only to software development (cf. Hibbs, Jewett, Sullivan, “The Art of Lean Software Development”).

At first glance, lean is a very good complement for agile. Users have concerns about agile’s ability to scale; lean makes sure that developer “resources” are available for each iterative step in a flexible and cost-effective way. Lean allows a different process time-line each time depending on what resources are available when; agile “spirals” in towards a solution, ensuring a different process timeline for each iteration of a solution “guesstimate”. Both aim to increase value to the customer; both emphasize incremental improvements.

However, a reading of the blogs on lean and agile betrays a fundamental misunderstanding of the agile mindset. Here’s a quote from Martin Heller (http://www.infoworld.com/d/developer-world/should-software-development-be-lean-or-agile-284): “They both strive to improve software quality, reduce waste, increase developer productivity, accept changes to requirements, and prize meeting the customer's real needs.” Agile doesn’t strive to improve software quality; it strives to improve software usefulness. It doesn’t aim to reduce waste; it aims to increase “wasteful” experimentation, which turns out actually to reduce the time-to-value, as a side-effect. Its way of increasing developer productivity – ensuring that development is more in sync with customer needs – is the opposite of lean’s attempt to improve developer productivity by eliminating bottlenecks (and customer interference is potentially a bottleneck). Lean accepts changes to requirements as a necessary part of incremental manufacturing process improvement; agile welcomes and emphasizes changes to requirements as part of improving product design. Lean views customer needs as consisting primarily of quality; agile views customer needs as consisting primarily of rapidly-changing functionality.

Moreover, we have already seen how an emphasis on Deming-like quality can retard product development to the detriment of a firm, as when Motorola lost its race to dominate processors to Intel by being late and customer-insensitive with the design of a “quality”-manufactured product. I can’t quote figures from my recent study under Aberdeen auspices of agile software development vs. other processes; but I can certify that a focus on software quality improvement was far less effective in improving product TCO, ROI, customer value, and ongoing agility than an agile process; in fact, there was no clear long-term benefit to ROI of quality-improving processes at all.

I wish I could say that this is a side-issue, and that once agile is properly understood, lean and agile are indeed complements. After all, some of the confusion about agile stems from a lean mindset that views everything from the lens of the manufacturing process. In many companies, integrating NPD and the manufacturing process has indeed reaped rewards, and lean has been a part of that improvement. Agile, by contrast, is effective at NPD and R&D/innovation, and will probably never be applicable to manufacturing – imagine asking the NC machine to “spiral in on” the correct product each time! Surely, a compromise can be worked out in which lean resource allocation is applied to agile software development and lean manufacturing becomes much more nimble at producing prototypes. However, I regretfully conclude that in the real world, mixing lean and agile is likely to produce worse results than applying agile alone.

Consider the idea of applying just-in-time resource allocation to an agile process. In each iteration, the agile process is going to change the specs during the “sprint.” Thus, half the time, allocation of resources will be inadequate, causing more bottlenecks than a “wasteful” strategy.

More fundamentally, lean increases the need for tight control over and visibility into costs and resources. This kind of oversight, as developers well know, slows things down. As per my study, it also yields nothing in improved productivity, TCO, ROI, or customer satisfaction.

Above all, the mindset of lean and agile are antithetical. Lean is reactive, prescriptive, and cost-oriented, despite its attempts to connect to the customer. It works where requirements change little from iteration to iteration, as in a manufacturing process. Agile is proactive, creative, and customer-value-oriented. It works where customer needs are constantly changing – which is a much better description of the real world of customers, as opposed to the artificial “turn off the spigot until we’ve finished” world of much of today’s NPD.

In the end, however, the conflict of lean and agile is only one instance of the difficulties companies have with processes like agile when the command-and-control approach has become entrenched. As I noted in a recent post, it may well be caused by budgeting processes that constrain business flexibility. But whatever the reason, there is strong cultural resistance to increasing corporate results long-term by adopting processes that increase business agility, such as agile development.

In an old story whose author I cannot unfortunately remember, a manager is called in to turn the efforts of R&D types into profit. The first thing he notices was that they seem to spend a lot of time with their feet on their desks, thinking – a highly wasteful practice. How can he change that? He puts his legs up on his desk and leans back, thinking … In this story, as in the real world, lean is more the enemy than the partner of agile; and trying to constrain agile via lean will cost the company much more in the long run than it seems to gain according to short-term cost measures.