The Transition From Agile Development to the Agile Organization Is Beginning to Happen

Disclaimer:  I am now retired, and am therefore no longer an expert on anything.  This blog post presents only my opinions, and anything in it should not be relied on.

Recently, new publications from CA Technologies via Techtarget arrived in my in-box.  The surveys mentioned in them confirmed to me that the agile organization or agile business is beginning to be a Real Thing, not just vendor hype. 

Five years ago, I wrote but did not publish a book on the evidence of agile development’s benefits and implications for creation of a truly agile business or other organization.  Now, it appears not only that the theoretical foundation has been laid for implementation at least of agile processes in every major department of the typical business, but also that a significant subset of businesses now think that they have at implemented agile according to that foundation across pretty much all of the enterprise – yes, apparently in a few cases including legal (what does legal-department agility mean?  I have no idea, yet).

So what are the details of this evidence?  And what benefits of an agile organization seem to be proving themselves?

The Solid Foundation of Agile-Development Benefits

It is now approaching a truism that agile development delivers benefits compared to traditional software-development approaches.  My own survey during my brief re-up at Aberdeen Group 9 years ago suggested improvements in the 20-30 % range for project cost and speed, product quality, and customer satisfaction (with the obvious implication that it also decreased product-development risk by around that amount, as Standish Group studies also showed).  One striking fact was that it was achieving comparable results when compared to traditional approaches focused on cost and/or quality.

One CA pub (“Discover the Benefits of Agile:  The Business Case for a New Way to Work) extends these findings to agile development plus agile project management.  It says that a “summary of research” finds that agile delivered 29 % improvements in cost, 50 % in quality, 97 % in “productivity” (something like my “speed”), 400 % (!) in customer satisfaction, 470 (!!) in ROI (a proxy for revenue and profit), compared to the “least effective” traditional approaches. 

While this may sound like cherry-picking, my research showed that the most effective traditional approaches were not that much better than the least effective ones.  So I view this CA-cited result as the “practice effect”:  experience with agile development has actually increased its advantage over all traditional approaches – in the case of customer satisfaction and profitability, by really large amounts.

The Theoretical Case For Business Agility

Note, as I did back when I did my survey, that agile development often delivers benefits that show up in the top and bottom line of the success-critical-software-developing business, even before the rest of the organization attempts to go agile.  So why would it be important to go the rest of the way and make most or all of the organization agile?

The CA pub “The State of Business Agility 2017” plus my own work suggest that potential benefits of “agile beyond software development” fall into three areas: 

1.      Hard-nosed top and bottom line benefits:  That is, effects on revenue, cost, and margin (profit).  For example, better “innovation management” via agile project management goes to the top line and eventually to the bottom line, and in some cases can be measured.

2.      “Fuzzy” corporate benefits, including competitive advantage, quality, customer satisfaction, speed to act and change strategies, and reduction in negative risks (e.g., project or IT-infrastructure failures) and “fire drills”.

3.      “Synergy” benefits stemming from most of the corporation being on the same “agile page” with coordinated and communicating agile processes, including better collaboration, better/faster employee strategy buy-in, better employee satisfaction through better corporate information, and better “alignment between strategy and execution.”

The results of the CA business-agility pub survey suggest that most respondents understand many but not all of these potential benefits before they take the first step towards the agile business.  I would guess, in particular, that they don’t realize the possible positive effects on combating “existential risks”, such as security breaches or physical IT-infrastructure destruction, as well as the effects on employee satisfaction and better strategy-execution alignment.

The Extent of Agile Organizations and Their Realized Benefits

Before I begin, I should note two caveats about the CA-reported results.  The first is that respondents are in effect self-selected to be farther along in agile-organization implementation and more positive about it.  These are, if you will, among the “best and the brightest.”  So actual agile-organization implementations “on the ground” are certainly far less than the survey suggests.

Second, CA’s definition of “agile” leaves out an important component.  CA’s project-management focus makes part of its definition of agility to be holding time and cost in a project constant while varying project scope.  What that really means is that CA de-emphasizes the ability to make major changes in project aims at any point in the project.  In the agile-organization survey, this means an entire lack of focus on the ability to incrementally (and bottom-up!) change a strategy rather than just roll out a whole new one every few years.  And yet, “more agile” strategy change is at the heart of agility’s meaning and is business agility’s largest long-term potential benefit.

How far are we towards the agile organization?  By some CA-cited estimates, 83% of all businesses have the first agile-development step at least in their plans, and a majority of IT projects are now “agile-centric.”  Bearing in mind caveat (1) above, I note that 22% of CA-survey respondents say they are just focused on extending “agile” to IT as a whole, 17% are also working on a plan for full business agility, 19% have gotten as far as organizational meetings to determine agility-implementation goals, and 39% are “well underway” with rollout.  Ignoring the question about “momentum” in partial departmental implementations for a moment, I also note that 47 % say IT is agile, 36% that Project Management is, and marketing, R&D, operations/manufacturing (are they counting lean methodologies as agile?), and sales (!) are a few percentage points lower. 

Getting back to partial implementation, service/support seems to be the “new frontier.”  Surprisingly, corporate communications/PR is among the laggards even in implementation, along with accounting/finance, HR, and legal.  What I find interesting about this list is that accounting and legal are even in the conversation, indicating that people really are planning for some degree of “agile” in them.  And, of course, the CEO’s agility isn’t even in the survey – as I said 9 years ago, the CEO is likely to be the last person in the organization to “go agile.”  Long discussion, not relevant here.

How about benefits?  In the hard-nosed category, agile organizations increase revenue 37% faster and deliver 30% greater profit (an outside survey).  For the rest of benefits, there is far less concrete evidence – the CA business-agility survey apparently did not ask what business benefits respondents had already seen from their efforts.  What we can deduce is that most of the 39% of respondents who said they were “well underway” believe that they are already achieving the benefits they already understand, including most of the “fuzzy” and “synergy” benefits cited above.

Implications:  The Cat Is In The Details

At this point, I would ordinarily say to you the reader that you should move towards implementing an agile business/organization, bearing in mind that “the devil is in the details.”  Specifically, the CA surveys note that the complexity of the business and cultural/political opposition are key (and the usual) problems in implementation.  And, indeed, this would be a useful thing to know.

However, I also want to emphasize that there is a “cat” in the details of implementation:  a kind of Schrodinger’s Cat.  In quantum physics, as I understand it, different states of basic particles (e.g., exists, doesn’t exist) are entangled until we disentangle them (e.g., by “opening the box” and measuring them).  Schrodinger imagined entangled states of “cat inside the box/no cat inside the box”, so that we wouldn’t know whether Schrodinger’s Cat existed until we opened the box.  In the same way, we not only don’t know what and how much in the way of benefits we get until we examine the implementation details, we won’t know just how really agile we are until we “open the box.”

Why does that matter?  Because, as I have noted above, the really big long-term potential benefit of business agility, is, well, agility:  The ability to turn strategically, instantly, on a dime and ensure the business’ long-term existence, as well as comparative success, much more effectively.  Just because some departments seem to be delivering greater benefits right now, that doesn’t mean you have built the basic infrastructure to turn on a dime.

And so, the cat is in the details.  Please open the box by testing whether your implementation details allow you to change overall business strategies fast, and then, if there is no cat there, what should you do?

Why, change your agile-implementation strategy, of course.  Preferably fast.

The Cult of the Algorithm: Not So Fast, Folks

Sometimes I feel like Emily Litella in the old Saturday Night Live skit, huffing and puffing in offense while everyone wonders what I’m talking about.  That’s particularly true in the case of new uses of the word “algorithm.”  I find this in an interview by NPR of Cathy O’Neil, author of “Weapons of Math Destruction”, where part of the conversation is “We have these algorithms … we don’t know what they are under the hood … They don’t say, oh, I wonder why this algorithm is excluding women”.  I find this in the Fall 2016 Sloan Management Review, where one commenter says “developer ‘managers’ provide feedback to the workers in the form of tweaks to their programs or algorithms … the algorithms themselves are sometimes the managers of human workers.” 
As a once-upon-a-time computer scientist, I object.  I not only object, I assert that this is fuzzy thinking that will lead us to ignore the elephant in the living room of problems in the modeling of work/management/etc. to focus on the gnat on the porch of developer creation of software.
But how can I possibly say that a simple misuse of one computer term can have such large effects?  Well, let’s start by understanding (iirc) what an algorithm is.

The Art of the Algorithm

As I was taught it at Cornell back in the ‘70s (and I majored in Theory of Algorithms and Computing), an algorithm is an abstraction of a particular computing task or function that allows us to identify the best (i.e., usually, the fastest) way of carrying out that task/function, on average, in the generality of cases.  The typical example of an algorithm is one for carrying out a “sort”, whether that means sorting numbers from lowest to highest or sorting words alphabetically (theoretically, they are much the same thing), or any other variant.  In order to create an algorithm, one breaks down the sort into unitary abstract computing operations (e.g., add, multiply, compare), assigns costs to each, and then specifies the steps (do this, then do this).  Usually it turns out that one operation costs more than the others, and so sort algorithms can be reduced to considering the overall number of compares n as n increases from one to infinity.
Now consider a particular algorithm for sorting.  It runs like this:  Suppose I have 100 numbers to sort.  Take the first number in line, compare it to all the others, determine that it is the 23^rd lowest.  Do the same for the second, third, … 100^th number.  At the end, for any n, I will have an ordered, sorted list of numbers, no matter how jumbled the numbers handed to me are.
This is a perfectly valid algorithm.  It is also a bad algorithm.  For every 100 numbers, it requires at least 100 squared steps, and for any n, it requires at least n squared steps.  We say that this is an “order of n squared” or O(n**2) algorithm.  But now we know what to look for, so we take a different approach.
Here it is:  We go through the list of 100 numbers from both ends, and we find the maximum of the numbers from the low end and the minimum of the numbers from the high end, stopping when both sets of comparisons are dealing with the same number.  We then partition the list into two buckets, one containing the list up to that number (all of whose items are guaranteed to be less than that number), and one containing the list after that number (all of whose items are guaranteed to be greater than that number.  We repeat the process until we have reached buckets containing one number.  On average, there will be O(logarithm to the base two, or “log” of n) such splits, and each level of split performs O(n) comparisons.  So the average number of comparisons in this sorting algorithm is O(n times log n), called O(n log n) for short, which is way better than O(n**2) and explains why the algorithm is now called Quicksort.
Notice one thing about this:  finding a good algorithm for a function or task says absolutely nothing about whether that function or task makes sense in the real world.  What does the heavy lifting of creating a new program that is useful is more along the lines of a “model”, implicit in the mind of the company or person driving development, or additionally explicit in the program software actually carrying out the model.   An algorithm doesn’t say “do this”; it says, “if you want to do this, here’s the fastest way to do it.”

Algorithms and the Real World

So why do algorithms matter in the real world?  After all, any newbie programmer can write a program using the Quicksort algorithm, and there are a huge mass of algorithms available for public study in computer-science journals and the like.  The answer, I believe, lies in copyright and patent law.  Here, again, I know somewhat of the subject, because my father was a professor of copyright law and I held some conversations with him as he grappled with how copyright law should deal with computer software, and also because in the ‘70s I did a little research into the possibility of getting a patent on one of my ideas (it was later partially realized by Thinking Machines).
To understand how copyright and patent law can make algorithms matter, imagine that you are Google, 15 or so years ago.  You have a potential competitive advantage in your programs that embody your search engine, but what you would really like is to turn that temporary competitive advantage into a more permanent one, by patenting some of the code (not to mention copyrighting it to prevent disgruntled employees from using it in their next job).  However, patent law requires that this be a significant innovation.  Moreover, if someone just looks at what the program does and figures out how to mimic it with another search engine set of programs (a process called “reverse engineering”), then that does not violate your patent.
However, suppose you come up with a new algorithm?  In that case, you have a much stronger case for the program embodying that algorithm being a significant innovation (because your program is faster and [usually] therefore can handle many more petabytes or thousands of users), and the job of reverse engineering the program becomes much harder, because the new algorithm is your “secret sauce”. 
That means, if you are Google, that your new algorithm becomes the biggest secret of all, the piece of code you are least likely to share with the outside world – outsiders can’t figure out what is going on.  And all the programs written using the new algorithm likewise become much more “impenetrable”, even to many of the developers writing them.  It’s not just a matter of complexity; it’s a matter of preserving some company’s critical success factor.  Meanwhile, you (Google) are seeing if this new algorithm leads to another new algorithm – and that compounds the advantage and secrecy.
Now, let me pause here to note that I really believe that much of this problem is due to the way patent and copyright law adapted to the advent of software.  In the case of patent law, the assumption used to be that patents were on physical objects, and even if it was the idea that was new, the important thing was that the inventor could offer a physical machine or tool to allow people to use the invention.  However, software is “virtual” or “meta” – it can be used to guide many sorts of machines or tools, in many situations; at its best, it is fact a sort of “Swiss Army knife”.  Patent law has acted as if each program was physical, and therefore what mattered was the things the program did that hadn’t been done before – whereas if the idea was what mattered, as it does in software, then a new algorithm or new model should be what is patentable, not “the luck of tackling a new case”.
Likewise, in copyright law, matters were set up so that composers, writers, and the companies that used them had a right to be paid for any use of material that was original – it’s plagiarism that matters.  In software, it’s extremely easy to write a piece of a program that is effectively identical to what someone else has written, and that’s a Good Thing.  By granting copyright to programs that just happened to be the first time someone had written code in that particular way, and punishing those who (even if they steal code from their employer) could very easily have written that code on their own, copyright law can fail to focus on truly original, creative work, which typically is associated with new algorithms.
[For those who care, I can give an example from my own experience.  At Computer Corp. of America, I wrote a program that incorporated an afaik new algorithm that let me take a page’s worth of form fields and turn it into a good imitation of a character-at-a-time form update.  Was that patentable?  Probably, and it should have been.  Then I wrote a development tool that allowed users to drive development by user-facing screens, program data, or the functions to be coded in the same general way – “have it your way” programming.  Was that patentable? Probably.  Should it have been?  Probably not:  the basic idea was already out there, I just happened to be the first to do it.]

It's About the Model, Folks

Now let’s take another look at the two examples I cited at the beginning of this post.  In the NPR interview, O’Neil is really complaining that she can’t get a sense of what the program actually does.  But why does she need to see inside a program or an “algorithm” to do that?  Why can’t she simply have access to an abstraction of the program that tells her what the program does in particular cases?
In point of fact, there are plenty of such tools.  They are software design tools, and they are perfectly capable of spitting out a data model that includes outputs for any given input.  So why can’t Ms. O’Neil use one of those? 
The answer, I submit, is that companies developing the software she looks at typically don’t use those design tools, explicitly or implicitly, to create programs.  A partial exception to this is in the case of agile development.  Really good agile development is based on an ongoing conversation with users leading to ongoing refinement of code – not just execs in the developing company and execs in the company you’re selling the software to, but ultimate end users.  And one of the things that a good human resources department and the interviewee want to know is exactly what the criteria are for hiring, and why they are valid.  In other words, they want a model of the program that tells them what they want to know, not dense thickets of code or even of code abstractions (including algorithms).
My other citation seems to go to the opposite extreme:  to assume that automation of a part of the management task using algorithms reflects best management practices automagically, as old hacker jargon would put it.  But we need to verify this, and the best way, again, is to offer a design model, in this case of the business process involved.  Why doesn’t the author realize this?  My guess is that he or she assumes that the developer will somehow look at the program or algorithm and figure this out.  And my guess is that he/she would be wrong, because often the program involves code written by another programmer, about which this programmer knows only the correct inputs to supply, and the algorithms are also often Deep Dark Secrets.
Notice how a probably wrong conception of what an algorithm is has led to attaching great importance to the algorithm involved, and little to the model embodied by the program in which the algorithm occurs.  As a result, O’Neil appears to be pointing the finger of blame at some ongoing complexity that has grown like Topsy, rather than at the company supplying the software for failing to practice good agile development.  Likewise, the other cite’s belief in the magical power of the algorithm has led him/her to ignore the need to focus on the management-process model in order to verify the assumed benefits.  As I said in the beginning, they are focusing on the gnat of the algorithm and ignoring the elephant of the model embodied in the software.

Action Items

So here’s how such misapprehensions play out for vendors on a grand scale (quote from an Isabella Kaminska article excerpted in Prof. deLong’s blog, delong.typepad.com):  “You will have heard the narrative.... Automation, algorithms and robotics... means developed countries will soon be able to reshore all production, leading to a productivity boom which leads to only one major downside: the associated loss of millions of middle class jobs as algos and robots displace not just blue collar workers but the middle management and intellectual jobs as well. Except... there’s no quantifiable evidence anything like that is happening yet.”  And why should there be?  In the real world, a new algorithm usually automates nothing (it’s the program using it that does the heavy lifting) and the average algorithm does little except give one software vendor a competitive advantage over others.

Vendors of, and customers for, this type of new software product therefore have an extra burden:  ensuring that these products deliver, as far as possible, only verifiable benefits for the ultimate end user.  This is especially true of products that can have a major negative impact on these end users, such as hiring/firing software and self-driving cars.  In these cases, it appears that there may be legal risks, as well:  A vendor defense of “It’s too complex to explain” may very well not fly when there are some relatively low-cost ways of providing the needed information to the customer or end user, and corporate IT customers are likewise probably not shielded from end user lawsuits by a “We didn’t ask” defense.

Here are some action items that have in the past shown some usefulness in similar cases:

·         Research software design tools, and if possible use them to implement a corporate IT standard of providing documentation at the “user API” level specifying in comprehensible terms the outputs for each class of input and why.

·         Adopt agile development practices that include consideration and documentation of the interests of the ultimate end users.

·         Create an “open user-facing API” for the top level of end-user-critical programs, that allows outside developers to (as an intermediary) understand what’s going on, and as a side-benefit to propose and vet extensions to these programs.  Note that in the case of business-critical algorithms, this trades a slight increase in the risk of reverse engineering for a probable larger increase in customer satisfaction and innovation speedup.

Above all, stop deifying and misusing the word “algorithm.”  It’s a good word for understanding a part of the software development process, when properly used.  When improperly used – well, you’ve seen what I think the consequences are and will be.

BigLever and the Agile Product Line

BigLever’s recent introduction of the Gears PLE (Product Line Engineering) Lifecycle Framework and PLE Methodology – adding to its long-time BigLever Software Gears offering for product-development management – is as important, I believe, for its potential ability to foster the “agile” products of the future as for its immediate benefit in extending product management to multiple inter-related product lines.  BigLever ably describes its offering’s immediate benefits to present customers.  I’d like to dwell here on the broader implications of its new capabilities in evolving software development to support not only agile development but also agile solutions.

What do I mean by an agile product/solution?  I mean one that is exceptionally and quickly customizable (not just modifiable as the product evolves) at all points in the product/solution lifecycle, both before and after its delivery to the customer.  In other words, agile development allows you to change as customer needs change right up to the point of delivery, and then iterate in the next product development lifecycle; agile products allow you to create product lines of multiple related products in which somewhere in the welter of choices is the right version for a “customer of one”, and that customer of one is a large company that needs multiple versions for multiple customers inside the firm.

I would argue that this is an inevitable outcome of today’s trends in software-infused products and product lines.  The marketing approach that seeks to leverage one successful product in related areas is now reflected in software-containing products that share a great deal of features, whether the results are presented to the outside world as one product line or several.  Increasingly, a marketing and Big-Data focus on customer individuality, and customers’ desire to achieve greater usefulness via customization after solution receipt, makes customization both before and after the sale more and more frequent. I would also assert that most of today’s software development tools assume the old reality of one linear product development lifecycle.  Yes, programmers and code are shared across related projects; but across multiple product lines, less so.

Step 1:  BigLever PLE and Cross-Product-Line Customization

Again from my point of view, the BigLever offering simply takes a project management/product-lifecycle-management framework and infuses it with the notion that multiple related products and product lines are involved.  That’s not as simple as it sounds; for example, doing change management on multiple related development streams is more complicated than doing branching of code versions in a single stream.  In any case, the BigLever software predefines a set of “variable” building blocks that represent features, and runs through a checklist for a particular version that determines which customer gets which feature set on which customization of which product – and allows for further post-production customization by the customer.

The rest is details, but important ones.  There is a traceability mechanism, a “product configurator”, a “tiered” Methodology (supporting products, product lines, and muitiple-product-line “portfolios”) and the ability to store code sets representing customizable features – a characteristic that effectively creates for the developer a base of industry- and firm-specific software from which he or she can semi-automatically populate a product line.

And it is this last detail that especially excites me, and leads me to talk about agile products.

The Other Shoe:  Automated Development With Embedded Customization

It seems to me that the ability to store a “code data store” of feature variations across products and product lines is much like the ability to create a library of object classes:  It forms a natural basis for rapid, flexible development.  Granted, Gears is focused on managing development across multiple projects, not on creating code itself.  Still, taking that code data store and using it to semi-automatically generate new software based on the old is an obvious next step for somebody.  And that new software retains the flexible customization of the previous version of a product, product line, or product “portfolio”, baked in, as it were. 

Consider such an approach when wedded to agile development.  Techniques such as refactoring already ensure that it is exceptionally easy to modify software as you develop it, as needed.  Building blocks with customization built in allow not just modification, but also variation/customization to fine-tune for the customer of one.  Together, they maximize the agile developer’s ability to go in any direction needed at any time.

That’s why I call the result an agile product, not a flexible one.  The distinction between flexibility and agility is key to getting an agile approach’s true benefits.  Flexibility can simply cement a sub-optimal solution in place, rather than encouraging major changes as customer needs change.  Agility, instead, encourages incremental adjustment that eventually add up to major new functionality, well adapted to the customer.  The combination of development agility in modifying existing software, and the kind of customization agility resulting from semi-automated development using building blocks with embedded customization, makes both the process and the result (the product) more agile.

Bottom Line:  The BigLever Approach is Worth Incorporating

Let me be clear:  BigLever does not force any approach on you. Its Gears solution has a long tradition of working with development toolsets of all stripes.  It is my own view that it or a similar solution will in the long run pay off best when paired with agile development.  And, of course, the details of the merger of the two have yet to be fully worked out.

Still, given the promise of the BigLever PLE approach, it seems reasonable to me that medium-to-large IT organizations with an array of software development needs that approximate “multiple related product lines” should at least take a close look at the new BigLever offering.  Savvy implementers might well go further, and begin to amass a “war chest” of feature customizations for later use in automated development.  In either case, the only thing you have to lose is an illusion:  The idea that we can go back to the good old days when each project aimed to create a new version of one and only one product.

But back then, you couldn’t even dream of the customer of one. Maybe those good old days weren’t very good, after all. You’re probably better off exploring the brave new agile-product-line world that BigLever’s offering represents.

 

Tiempo Development and the Agile Business

I have heard a future of business organization, and it sounds as if it works.

After ten years of hearing companies only begin to appreciate agile development and then agile marketing, I certainly was not expecting to hear a company organize its management, its strategy, and its planning around a company-wide agile process. But when Tiempo Development, billing itself as yet another agile development outsourcer, gave me a telebriefing last week, I asked the obligatory question and was floored to hear that they had, indeed, done so. 

The Beginning Of The Epic

I am sure that, like many another agile-development startup, Tiempo asked themselves why they couldn’t apply agile principles to management – and yet, others I have talked to stopped there.  Tiempo, however, when faced with the need to begin doing better planning and budgeting as the business scales, seems to have had the guts to take a leap of faith and try to do it via a variant of Scrum, complete with Kanban-type lean scheduling, plus the “epics” and “stories” beloved of agile marketing.  In other words, there’s a five-year plan and budget (and, of course, in an agile organization that’s not set in stone) devolving into yearly, quarterly, monthly, and weekly epics and stories. 

Revisions to the plans are quarterly, flowing forward into a new rolling five-year plan. As in Scrum, daily task sessions and weekly reviews communicate widely, and change flows from below as well as above.  “Customer” feedback comes in laterally and frequently from project customers, as well as from above and below on the plans and their implementation.  To put it another way, the concentrated but broad communication of Scrum ensures the maximum of customer-driven evolution, while the five-year horizon ensures that long-term strategic considerations engage in a delicate dance with short-term customer –driven product specs.

We are long past the time when companies would ask whether agile development scales.  Still, the question should be asked:  how well does a Tiempo-style “Scrumban” management approach scale?  The apparent answer after about 1 1/2 years of practice is:  Quite well, so far.  The reason, I would suggest, lies in the initial establishment of five-year company goals in terms of “rocks”.  That is, the implementing CEO picked about five goals that formed what he saw as the fundamental things that the company needed to achieve over the next five years – the “rocks” in both the foundational and the risk sense. 

Again, as noted, the “rocks” are not made of concrete; they evolve over time, fairly frequently, as the company evolves.  Think of them as “virtual rocks”.  They therefore combine long-term perspective; frequent review based on customer and employee feedback; and a broad perspective, not only for the CEO but also in terms of employee understanding of (and buy-in to) strategy. Plug in more business, plug in new areas, plug in new employees.

To Understand Agile Management’s Effectiveness, Redefine Success

What does Tiempo report as evidence of success in agile management?  Interestingly, they start first with better communications between managers and “line” developers, followed by quick surfacing of bad news for rapid handling, and then “empowering” the employees.  It was only when I probed deeper that the follow-on effects emerged:  more rapid development without other sacrifices, leading to better profitability; greater customer satisfaction, leading to greater follow-on business and willingness of large enterprise customers to trust Tiempo in new areas; and a strong growth path that has the CEO more concerned with avoiding lack of focus via too many new areas than with short-term risks.

It was not a given that this would be so, but in this respect agile management is mirroring agile development.  My survey four years ago found a persistent pattern of comparative focus on change rather than cost, speed, profit, quality, or (downside) risk leading to greater cost, speed, profit, quality, and customer satisfaction as well as “upside” risk, while “downside” risk decreases – all compared to attitudes and processes that do focus on these things.  That Tiempo is focusing on the employee and customer satisfaction parts of the effect is what we should expect if Tiempo is “talking the agile talk and walking the agile walk.” And so, if agile management is indeed like agile development, out of this process we should indeed expect better top-line and bottom-line results that are likely to endure.

The problem is, of course, metrics that will establish these things – because the merits of agile management are not as easy to prove as those of agile development.   A fluke, a Board of Directors and the stock market will call the first agile management bottom-line results, caused by unique circumstances and remnants of past practices.  All these daily and weekly meetings, are they really doing any more than past business strategy fads?  I believe strongly that the answer is resoundingly yes. But I can’t do a survey like the agile development one to prove the point, because there are no obvious speed, cost, and risk-reduction metrics for the CEO’s actions, just customer and employee satisfaction, which have proven misleading in the past.

And so, we are back to CEO “culture”, just as we had to deal with IT and business management “cultures” that were highly resistant to the touchy-feely feel-good aspects of agile development.  Remember, CEOs are the byproduct of a selection process that emphasizes enormous, visible hard work and schmoozing as well as hard-nosed attention to the bottom line, and rewards these by surrounding the CEO with those who understand that their future depends on giving the CEO positive reinforcement, as well as with compensation that most fair-minded outside observers concede is well beyond the demonstrated relative effectiveness of the CEO.  In other words, power and money.  I once took a survey at B school that asked whether I wanted to make money, have power, do good for people, or accomplish something; it turned out that those who answered power or money got much more of both.

I mention this because (and this is only my impression) what really got the CEO excited, talking to me, was his own personal satisfaction – that is, the “accomplish something” part of the B school survey.  And yet, there was no reason he wouldn’t wind up with just as much power and money as my classmates who answered “power” or “money”.  In other words, in the agile world, you can make as much (or more, since the organization should be more successful) money and have as much power, but work less (remember, you’re not running around trying to single-handedly move an ocean liner of an organization or sell the world on your product) and get more real satisfaction out of it, because you can point to shared concrete projects finished successfully, not just praise from those who wish to share in your money.  And so, if the CEO and upper management can make the cultural shift, they may well find that they are better off in every way, including personal satisfaction – another agile paradox.

In fact, it turns out that the creators of agile processes may have been craftier than they knew in calling projects and project groupings “stories” and “epics”.  It speaks to something Tolkienesque in all of us.  A study recently reported in MIT Technology Review suggests that we don’t have fixed memories like snapshots of events, we have stories of the past, and each time we revisit them, we can change them, e.g., to fit new understandings of the past, a new narrative arc.  When I do TCO studies, it’s not all about yes/no, 1-5, and numbers; I ask the interviewee to tell his or her story, the story of him or her in the project.  Set the stage for me, what was your environment; tell me how the project unfolded; tell me what you found were the results; and only now tell me how this affected the TCO that you find justifies the project.  I have found the insights from such an approach far richer and better founded than those from standard surveys, even when the data is sparser.

In an agile business such as Tiempo’s, it sounds as if everyone becomes a hero or heroine in a shared epic that never ends, always new and unfolding.  Oh, and by the way, it should perform better than an old-style business.

Caveats and Not So Bottom Line

I have always been a bit dissatisfied with the analyst’s usual disclaimer:  Your mileage may vary.  Yes, and your hair may grow longer or get cut shorter.  In this case, my real disclaimer is not that agile management may or may not apply to your organization, but rather that I don’t have the figures or real-world use cases to support my belief that Tiempo’s agile-management experience applies well to most medium-sized and large-sized enterprises and organizations. 

Will Tiempo continue scaling?  Does the agile management model appropriate to a small-to-medium-sized software-development outsourcer apply to most other types of business – not the traditional “hardware” manufacturing firm that forms the “unconscious” of all businesses today, and which as far as I can tell is a small fraction of today’s businesses, but the rest of them?  What I see and hear says that it should; but until it happens, there are afaik no decent metrics to show it is working.  Your mileage will probably not vary, but you won’t know the mileage for sure until the end of your first long trip.

No now we can proceed to my not so bottom line – because, remember, agile is about focusing on change, and the bottom line will follow.  Agile development, by now, has thousands of successful use cases. New product development is getting there, and there is now a “hardware” (silicon foundry) use case, as cited in a blog post a while ago.  Agile marketing is approaching its thousandth use case – I note that in a little over a year, the Bay Area agile marketing group has gone from zero members to well over a thousand; and IT is doing its best to keep up.  And now, we have an agile management – and hence agile business – use case. It’s time to start shifting the burden of proof.  Instead of wondering why you should be an agile developer/marketer/business, you should be demanding good reasons why you shouldn’t be.

I have heard a future of business organization, and it sounds as if it works.   Please stop asking about the mileage and start installing the tires.

Agile Marketing: The 40,000-Foot View

After learning about some of the fascinating efforts that the “agile marketing group” are making to have a process corresponding to “product development agility” over the last month, I find it’s a good time to set down some possibly useful thoughts on the subject. These are thoughts from the point of view of a computer industry analyst reasonably well versed in software development and agile concepts in that area, and with a reasonable theoretical and consulting acquaintance with marketing to businesses.

I call this a 40,000-foot view because I think that the success of agility in product development should challenge marketers to rethink their jobs fundamentally, not incrementally. However, it is all too easy to take a 20,000-foot view as from an airplane, seeing the broad outlines of marketing as it is today and picking and choosing those parts of product development agility (like the notion of “simplicity”) that comfort rather than challenge the marketer. With a 40,000-foot view, only the largest of mountains breaks up the monotony of land or sea, and we avoid getting bogged down in details or transitory marketing practices.

The aim of this view, therefore, is to think:

·         Long-term;

·         In terms of customer lifecycles;

·         Focusing on customer change rather than a succession of static preferences.

I hope to end up asking how the agile marketer can steer company products and services for the most effective “customer dance” with a broad range of customers, and what kinds of agile practices drawn from development can best serve them in this effort. But first, alas, definitions.

A Different Marketing Model

First, I would suggest that, as I noted in a previous piece, all “customer dances” (ongoing interactions with a customer) can be thought of as offering the company’s “worldview” to the customer to adopt or not, in competition with other companies’ worldviews.  For example, a buyer of insurance is offered several alternative ways of entering into a world of protection from nasty things that can happen to the customer. Or, a buyer of video games is offered many ways of living in alternate worlds where certain types of skills are trained for. If one thinks of a typical adult customer’s day, it starts with products and worldviews associated with home, then with worldviews that aid one at a particular type of job, then worldviews associated either with entertainment or more home tasks.

I suggest that customers’ needs for worldviews can be vaguely sorted into three categories:

1.       Learn. There are psychic as well as physical rewards for people who learn. Moreover, it is an open-ended process, one that need never end. Examples might be education, hobbies, training, passions, and fantasies.

2.       Do. This has been the traditional realm of much marketing, and has traditionally been thought of as mostly involving survival, job, or family support needs, ranging from buying food to mowing the lawn.  This is often true; but often there are psychic rewards for just accomplishing something “well”.

3.       Interact.  Pure Learn and Do are solitary activities; pure Interact has no purpose other than connecting to other people. This recognizes both the need for and psychic rewards for these connections. Things like phones, email, meeting halls, and dates are, by and large, mostly for Interaction.

It should be obvious that much of what is sold is a mixture of all three; but it is frequently useful to ask which of the three predominates.  For example, a PC today is typically used in the consumer market predominantly to Learn (the blogosphere is an example) and secondarily to Interact, while in the business market it is primarily used to Do.

The marketer therefore has two fundamental choices:

1.       Which worldview – which mix of learn, do, and interact – should I associate my company (brand, products, services) with? In other words, the choice is not merely one of industry – if “marketing myopia” didn’t make that clear – but also of the way the company would like the customer to think about the world (obviously, one that will sell lots of stuff) – bearing in mind that the customer has things to do and lots of alternatives, and so the time the customer spends in any one company’s worldview is very limited.

2.       Where in the customer’s life should I begin my relationship with the customer, and where should I end it?  Let’s face it, babies don’t buy products by themselves, and neither do the elderly in assisted living.  The relationship with the customer must have a beginning and an endpoint. Call this the relationship lifecycle.

Coming to Different Agile Answers

Before we launch into suggested “best” choices, let’s pause and think of ways that agile marketing might be said to be different from other marketing approaches.  If I had to give a short answer, it would go something like this:

·         Agile marketing emphasizes frequent, two-way conversations with the customer. In other words, the company does not decide a worldview and relationship lifecycle in isolation:  It constantly evolves both in partnership with its customers.

·         Agile marketing increases the company’s own emphasis on learning, especially compared to doing.  That change does not force the company to adopt a worldview that emphasizes learning more than before; but it does give the company additional ability to implement a worldview oriented more strongly towards customers’ “learn” needs. Thus, for example, Amazon.com can learn customers’ book preferences and trade that for increased sales by giving customers advance notice of publication of books they are fond of.

·         Counterintuitively, an emphasis on iterative, incremental new marketing and customer interactions leads agile marketers to a more long-run view of the customer relationship. Tighter bonds with customers lead naturally to greater efforts to extend the relationship.  That, in turn, gives the company new capabilities for longer relationship lifecycles.

·         Finally, compared to previous marketing approaches, agile marketing emphasizes more the idea that change is good. Mostly, this refers to changes in company marketing practices, company attitudes towards customers, and changes made during marketing projects.

Now let’s think selfishly about what a company wants, in order to maximize revenues/profits in the long term.  The company wants:

·         The most possible (profitable) customers;

·         Over the longest possible relationship lifecycle;

·         With the greatest customer loyalty (here I mean not just continuing to buy the same products, but also following the company into new products/services).

Let me pause for just a moment and put that in terms of worldviews and lifecycles. Here’s a grossly generalized Table:

                 Worldview                              Relationship Lifecycle

Learn       Values trading info                Very long

Do             Values quick solutions         Short

Interact     Values lots of interacters    Medium-term

And so, agile marketing appears to improve a company’s revenues/profit maximization in three ways:

1.       It lengthens the company’s typical relationship lifecycle by emphasizing Learn customers more.

2.       It increases customer loyalty by both involving the customer more and providing follow-on or related products/services more rapidly.

3.       It increases the number of customers both by speeding company movement into related markets and by decreasing customer disloyalty.

4.       It makes the company itself more able to change its target customers and therefore itself as the environment changes in future – by helping to make the company culture comfortable with change.

Two Personal Views: Addicts and Learning

I believe that agile marketing should also consider two other long-term aspects of customer “needs,” because, in the long run, I see traditional company strategies as sometimes harmful to companies’ long-term success.

The first aspect is the company-strategy tendency to sell to what I call the “addict” – someone who is or becomes psychologically addicted to a type of product or service.  Obviously, there are food addicts and drink/drug addicts; but there are also shopping addicts, home-repair addicts, conversation addicts, and even flying addicts. The characteristic of the addict is that he or she is impelled to spend more on a particular product/service type than is good for him/her – but that’s the type of customer that a company reflexively wants.  You want to buy?  Buy more!

I would argue two things about the addict.  First, in the long run, feeding addicts shortens relationship lifecycles (the addict’s life goes down the tubes), decreases other-customer loyalty (all I hear from you is buy, buy), and decreases revenues from the customer (I’m in hock up to my eyeballs). Second, the Learn customer is least likely to be an addict – both because the customer himself/herself is most adaptable to the changing needs of the workplace, and therefore more likely to have cash to spend, and because the customer may be more “realistic”, and therefore more likely to recognize and avoid addiction.

Therefore, I conclude that marketing in general, and agile marketing in particular, should consider the company’s present mix of addict and non-addict, and shift towards fewer addicts, possibly by moving towards a greater mix of Learn in its customers.

The second aspect of customer needs I think agile marketing should consider, irrespective of addiction, is which of the three “types” of customer would be best for the company, say, 10 years from now. We can point to plenty of companies that emphasize Learn, like video-game companies, and plenty that emphasize Interact, like the cell-phone and social-media companies, and definitely plenty that continue to emphasize the Do customer.

I would argue that most consumer companies, at the least, would benefit by increasing their proportion of Learn customers. They appear to be least likely to be addicts; they are least likely to get tired of or get a surfeit of the stuff provided; they have the potential to offer very long relationship lifecycles; they can provide a surprising amount of valuable information to the company along with their money; and the newness of what you provide is generally more exciting, while being “safe”. And since most B2B companies wind up providing value-add to products and services that wind up in the hands of consumers, they, too, should consider whether to improve their Learn customer value-add as passed on by the intermediate business.

The Agile Marketer’s Job

Implicit in the above analysis is a fairly simple statement of what should be the agile marketer’s general strategy:

Stretch all dimensions of the customer’s worldview/lifecycle.

This means, specifically:

·         Try to make the customer see you as a larger part of their life, by seeing you as in sync with their life and offering an interesting way to approach more and more of that life.

·         Try to stretch the length of the customer lifecycle backward and/or forward, by making the “cool stuff to learn about” accessible at earlier ages and applicable at later ages. Is it home repair you’re selling, or new cool solar ways to slice and dice your dwelling flexibly and in line with your own unique fashion sense – even in doll houses or Sims?

·         Increase customer loyalty to existing products/services by “crowd-customer-sourcing” their frequent upgrade, like bees building a hive under your (the queen’s) direction.

·         Increase customer pull to follow-on products by involving them early, focusing on Learn-type products/services/messages (build a better Lego Star Wars game!), and measuring/interacting two-way with customers rather than relying on the CEO’s “opinion.”

·         Make the brand a “learning” one (we’re in this together with the customer, and we’re going to do something insanely great), and drive comfort with “agile marketing” through the rest of the company by incorporating the idea of being agile in the company brand and culture.

I would also, myself, add three other elements of that strategy:

·         Both in product and promotional marketing, act as an integrator and not just as a coordinator of the rest of the company.  Coordinators make sure the functions don’t stumble over each other; integrators put “all the wood behind one arrow”, as Scott McNeally of Sun used to say, everything behind one overall strategy and consistent with one overall brand.

·         Stretch towards transparency to your customers and sacrifice some of that secrecy from your competitors as a tradeoff. Amazon wins because it always knows more about the evolution of its customers’ book needs, not because it hides its new software from its competitors. Moreover, as I have argued elsewhere, coopetition plus agility is a better long-run strategy than cutthroat competition.

·         Emphasize agile marketing software infrastructure. That is, pick a set “tools that believe”, tools that aim to foster marketing process agility or “get out of the way”, to support the marketing process and customer communications. Collaborative new-product development tools that link in the company’s Web customer “community” are one example of this.

Nasty Problems

Changing company culture rarely occurs without serious resistance (is that enough of an understatement for you?). And that is where the problems with applying this 40,000-foot view of agile marketing strategy to a ground-level business will typically lie. A frequent theme of the recent agile marketing manifesto “sprint” was seemingly unavoidable resistance from the rest of the company – or even other marketers.

Obviously, without enough experience in those situations, I can’t supply a definitive answer.  I can, however, suggest a couple of strategies:

·         In any in-business confrontation, greater knowledge of the customer is frequently a trump card. Specifically, if you say “surveys show that this is the way we are coming across to the customer”, that is a very good long-run argument when you either “kick the argument upstairs” or face an inevitable “shoot-out at high noon.”

·         Most people in business always have a yearning for “safe change”; why else would higher-ups enjoy golf, which always involves getting yourself to learn more and more about how to get better and better, without any real risk to the rest of your life? However, that has typically been ground under by the so-called realities of making a profit by getting a fixed thing done on time and on budget.  See if the resister in question can get the idea that agile is “safe”, that he or she will get it done faster and better by going agile, and that there is really less risk involved in continually trying something new, the “agile” way.  As many have testified in the past, once people get that idea, their enthusiasm and their agile performance is better than before.

Present-Day Agile Marketing Approaches to Build On

For a hotbed of agile marketing ideas, I’d suggest going to the Agile Marketing Facebook Group first. Here’s a snapshot of some of their “values” … :

1. Validated learning over opinions and conventions.
2. Customer-focused collaboration over silos and hierarchy.
3. Adaptive and iterative campaigns over big-bang campaigns.
4. Process of customer discovery over static prediction.
5. Flexible planning vs. rigid planning.
6. Responding to change over following a plan.
7. Many small experiments over a few large bets.

And “principles”:

• Simplicity is essential.
• Learning via a build, measure, learn feedback loop is the primary measure of success.
• Sustainable marketing requires you to keep a constant pace and pipeline.
• Don't be afraid to fail; just don't fail the same way twice.
• Continuous attention to marketing fundamentals and good design enhances agility.
• Deliver marketing programs frequently, from every couple of weeks to every couple of months, with a preference for a shorter time frame.
• Great marketing resources requires close alignment with the business people, sales, and development.
• Build marketing programs around motivated individuals, give them the environment and support they need, and trust them to get the job done.

Conclusion:  Movements, Agile Lemonade, and Pay for Fun

Let me close this 40,000-foot view by giving two additional thoughts possibly useful for agile marketers.  First is that the agile marketing effort bears every stamp of what in the computer industry is sometimes called a “movement.”  What’s a movement?

I define a movement as a new kind of market, or way to grow a market, characterized by:

·         Enormous, even quasi-religious, enthusiasm.

·         Participants who act as both initial vendors and initial consumers.

·         Ability to judge offerings not just by cost and features, but also “more [agile]” or “less [agile]”.

·         “Tools that believe” – software infrastructure that embodies the principles (in this case, of agile marketing).

Past examples of movements in the computer industry include Unix, open source, Linux, the Web – and agile software development.

To foster the success of a movement, I think, is a matter of:

·         Encouraging rather than “taming” the enthusiasm.

·         Publicizing the eventual, inevitable large-scale success (P&G?) when it occurs.

·         Moving rapidly to “tools that believe” and small-scale consultants so that new users see it as “safe” to try agile marketing.

And so, I think that agile marketers should join, contribute to, and use the resources and proof points of the agile marketing movement.

My second thought is that I find, in specific situations, “thinking agile” as a strategist involves taking things presented as problems and thinking of them as opportunities: “Getting lemons and making agile lemonade.”  Worried about hacker attacks? Why not design security software to place potential customers in a “sandbox”, and then trade knowledge of them for knowledge of you as a means of determining their bona fides? The hacker will betray himself/herself by his/her desire to access valuable company information without offering equivalent personal information in return – like location.  The valued customer will readily trade appropriate new information about himself/herself, enriching your ability to serve him/her in future. Problem? Opportunity!

And now, I’d like to close by reminding agile marketers of a key reason why the game is worth the candle:  Users report that agile makes you feel good because you are constantly learning and then doing better.  Here’s a story from the late 1990s, in the glory days of computer industry analysts:  I was sitting around with a fellow analyst at Aberdeen Group and we were talking about what was the real reason we didn’t chuck it all and go join some cool new Web startup like Webvan.  “You know,” he said to me in a tone of amazed happiness, “They’re paying us for learning!”

So there’s at least one good reason for agile marketers’ enthusiasm. Taking a 40,000-foot view, learning is where marketing should be headed – and it’s not just going to be good, it’s going to be fun.