Agile—the business philosophy that relies on fast-moving, self-managing teams for innovation—has officially entered the mainstream of corporate management. Tour almost any large company these days and you will find scores of agile teams working to improve customer experiences and business processes. John Deere has used agile methods to develop new machines, USAA to transform its customer service, and 3M to run a major merger integration. Bosch—a global supplier of technology and services with more than 400,000 associates—has adopted agile principles to guide a step-by-step reshaping of the company. Digital natives such as Amazon, Netflix, and Spotify have incorporated agile methods into a wide range of innovation activities. Meanwhile, agile has virtually taken over IT departments, themselves the source of countless innovations. At last count, 85 percent of software developers use agile techniques in their work.1
The reasons for agile’s rapid spread are neither obscure nor surprising. Most big companies find it difficult to innovate. They are weighted down by the structures and procedures of bureaucracy. Agile liberates the innovative spirit that so many organizations stifle. It helps companies reshape both what they offer their customers and how they operate internally. It transforms the work environment, making people’s jobs more rewarding.
These are grand claims, but the data support them. Study after study find conclusively that agile teams are far more successful at innovation than teams that work in traditional fashion. Improvements come rapidly and at less expense. Satisfaction and engagement among employees rise. Moreover, companies can implement agile without the need to spin off separate business units or hide skunkworks from hierarchies. They can deploy agile teams in any business or function that might benefit from them, including corporate headquarters. Once they have learned the basics, they can scale agile, establishing hundreds of individual teams or teams of teams to tackle large projects. Saab’s aeronautics business created more than one hundred agile teams operating across software, hardware, and fuselage for its Gripen fighter jet—a $43 million item that is certainly one of the most complex products in the world. Military analyst Jane’s has deemed the Gripen the world’s most cost-effective military aircraft.
So agile is spreading, and agile teams are mostly achieving their objectives. It looks like encouraging progress toward an appealing vision. Is there anything wrong with this picture?
Certainly there’s nothing wrong with the basic idea. We are business consultants, and we have seen the power and potential of agile in hundreds of companies around the globe. We have helped many of these companies implement agile. We count ourselves among its biggest fans.
But as with so many good ideas, the practice sometimes belies the promise. Agile has spread so rapidly that it threatens to spin out of control. Along with the companies that use it effectively are those that misunderstand or misuse the ideas. They may be egged on by some zealot who promises the world. They may sign on to an agile transformation before they know anything about what such an effort might entail. They may use agile terminology to camouflage distinctly nonagile objectives.
The outcome of these misuses in many companies is chaos rather than constructive change. But the damage is greater than any one company’s experience. When agile is done wrong, it almost always leads to lousy results. Lousy results lead to nervous customers, dissatisfied employees, activist investors, and a push to replace the management team. Replacement managers are understandably skeptical of any strategies that got the prior regime fired. They are likely to clean house, disband agile teams, and (probably) launch a round of layoffs. It’s a version of Gresham’s law: bad agile drives out good. If that happens too often, agile will be discredited—and the business world will be back where it started, with top-heavy bureaucratic corporations struggling hopelessly to keep up with brash upstarts and rapidly changing markets.
So in this book we want to bring agile down to earth, to separate Agile Done Right from Agile Done Wrong. Here, in the introduction, we’ll focus on the wrongheadedness, the potholes and pitfalls, the ways in which companies have already misunderstood or misused agile. We hope that the lessons and cautionary tales will inoculate you against the idea that agile is some kind of magic quick fix. But we’ll also introduce some of the ideas that inform the chapters that follow—chapters that will tell you how to do agile right. We’ll provide a road map to these chapters, and we’ll summarize the research that underlies the book. Doing agile right may take more time and experimentation than doing it wrong—but it’s the only way to get the results that the philosophy promises.
Doing Agile Wrong
In the movie The Princess Bride, the swordsman Inigo Montoya famously chides the cunning Vizzini, “You keep using that word. I do not think it means what you think it means.” So it is with agile. Executives often fail to understand how agile operates and where and why it has succeeded. That doesn’t stop them from throwing around the terminology or from making assumptions about agile that simply aren’t true.
Some of these misunderstandings reflect the fact that agile methods—especially those relating to expanding the scope and scale of agile innovation teams—are still relatively new, and many business leaders haven’t yet learned much about them. It’s common to hear, for example, that agile is great but only for technology-based innovations and the IT departments that generate those innovations. This will come as news to National Public Radio, which used agile methods to create new programs; to the people developing the Gripen fighter jet or Haier’s home appliances; and to the many companies using agile to reshape their supply chains. Historically, to be sure, agile has spread most rapidly in IT. But it is widely and successfully used in many other contexts, some of which have only minor technology-based components.
Other failings, though we hate to say it, reflect a measure of cynicism on the part of corporate leaders. Consider the clever press release that Edward S. Lampert, CEO of Sears, issued in 2017: “In addition to the cost reduction target announced today, we continue to assess our overall operating model and capital structure to become a more agile … and innovative retailer focused on member experience.”2Agile in this context is a euphemism for layoffs. And Lampert isn’t alone. Every month we receive more requests for proposals that begin something like this: “The project objective (should you choose to accept it) is to reduce operating expenses by 30 percent this year while transforming the organization to agile ways of working and digital technology.”
The issuers of these requests don’t understand that there are basic inconsistencies between large, chaotic layoffs and agile. For one thing, large layoffs tend to happen in batches driven by precipitous restructurings or annual budgeting cycles. This is diametrically opposed to the continuous learning and adaptation processes prescribed by agile. For another, senior executives typically go behind closed doors to plan the layoffs, emerging with new structures and fixed targets. This is inconsistent with the agile principle of empowering people closest to the work to identify improvement opportunities. Worst of all, leaders who try to marry agile with layoffs inadvertently role-model antiagile behaviors. They create predictive, command-and-control events rather than agile, test-and-learn cultures. Moreover, research shows that large layoffs increase risk aversion and slow innovation. People scramble to learn new jobs. They battle for control of key operations, no matter what the organization chart says. They do everything possible to make sure they will have a chair next year, when the music is likely to stop once again. Mostly, they try to do the same things they know how to do, but with fewer people. This is not an environment in which agile can flourish.
But then there is a different kind of wrongheadedness, not generated by complete ignorance or cynicism. These misuses are propagated by well-meaning agile partisans. They are sold to leadership teams who badly want their companies to become nimbler and more innovative but who don’t really understand how agile operates. In our work with hundreds of companies launching thousands of agile initiatives, we commonly find three toxic mistakes.
Agile, Agile, Everywhere
Some agile gurus pitch the approach as a panacea that must replace bureaucracy everywhere—in every company, in every business unit, in every function.
Consider a company that we’ll call MagicAgile. (This is a real company, but we don’t want to identify it because our conversations with its leaders were confidential.) The people at this company wanted to be like digital disruptors such as Spotify, the music streaming company that is well known for its agile innovation teams. So MagicAgile launched agile teams throughout its organization. It redesigned office spaces to create more open work areas. It innovated around customer and employee experiences. Through the eyes of agile evangelists, MagicAgile looked like a poster child—or at least it did if you ignore the fact that it lost about half of its market value in 2018 and early 2019 (or if you can claim with a straight face that things would have been even worse without the transformation, or that shareholder value doesn’t matter anyway). But in candid meetings with the leadership team of MagicAgile, we heard nearly every executive voice frustrations with their less visible results. They said things like this: “Agile has created a leadership problem. We have no discipline or alignment. This is chaos.” “We went too far. Everybody is talking about servant leadership and psychological safety. Nobody is allowed to use the word manager, and all managers have gone into hiding.” “Responsibilities for P&Ls are getting confused.” “Our leaders are criticized and ignored when we try to provide strategic guidance to our business units.” “Agile became the objective. We are praying at the altar of a false church.”
What the agile-everywhere zealots fail to understand is the time-tested virtues of bureaucracy—in some places and contexts. The same bureaucracy now so widely reviled as antithetical to change and innovation was itself one of the greatest innovations in the history of business. Hierarchical authority, specialized division of labor, and standard operating procedures—the hallmarks of the bureaucratic method—enabled companies to grow far larger than they had ever been. The principles of bureaucracy were taught as good management practices in business schools and corporate training programs. Companies learned the virtues of predictability and planning. Strong bureaucrats rose to the tops of their organizations.
Today we understand bureaucracy’s limitations. The great German sociologist Max Weber, who was the first to offer a systematic description of bureaucracy and who well understood its efficiencies, famously warned that it could create a soulless “iron cage” that trapped people in dehumanizing organizations and limited their potential.3 He was right: most people today work in bureaucracies, and most feel disengaged from their work. The widespread appeal to young people of start-ups and small businesses, as compared with careers spent climbing corporate ladders, reflects this flaw. Then, too, bureaucracies are terrible at innovating. A bureaucracy works when its organizational tasks—what to deliver and how to deliver it—are clear, stable, and predictable. Innovation, by definition, meets none of these criteria. These limitations have contributed to bureaucracy’s bad reputation and to the growth of antibureaucratic approaches such as agile.
And yet. Consider the adverse consequences of encouraging wide variation, on-the-spot experimentation, and decentralized decision making—all hallmarks of agile—in areas such as food or drug safety, antidiscrimination and harassment policies, accounting standards, aircraft safety, quality controls, and manufacturing standards. Every company has to run its business, getting standardized products out the door and delivering predictable services to customers. Every company needs bureaucratic structures and procedures, including hierarchical approvals, specialized division of labor, and standard operating procedures, to do exactly that.
The challenge, in short, is not to replace bureaucracy with agile everywhere but to find the right balance between the two. Every company must run its businesses. It must be good at operations. Every company must also change the business, continuously introducing not just new products and services but new operating methods and procedures. It must be good at innovating. Although these tasks require different skills, they are not enemies. They are complementary, interdependent, mutually beneficial capabilities that need each other to survive. Insufficient focus on innovation leads to a static enterprise that will fail to adapt to changing conditions. Insufficient emphasis on operations creates chaos—poor quality, high costs, and dangerous risks to customers and to the business.
Most large companies today have tilted too far toward bureaucracy, starving innovation. They have created static organizations committed to delivering predictable results. This is why agile is so popular. But the solution isn’t to tip the scale all the way in the other direction. Rather, it is to stick with bureaucratic rules and hierarchies where they are appropriate, humanizing them as much as possible, and at the same time to instill a healthy admixture of agile wherever it is appropriate. That may sound simple, but it isn’t. Agile and bureaucracy are like oil and vinegar: they are good together, but they don’t mix easily. (Sometimes the two act more like nitric acid and glycerol: they lead to explosions.) Agile teams thrive on doing things quickly. The teams try out new ideas, often before the ideas are fully formed, and test them with prospective customers. They don’t respect red tape, and they don’t follow detailed plans. If such teams are to thrive in an organization, they need a lot of freedom and a lot of support. Bureaucracies, of course, are just the opposite: they thrive on—indeed, require—tight control. They want to know exactly what a team has done so far, what it plans to do for the next twelve months, and how much it is all going to cost. To a traditional bureaucracy, agile teams can feel like foreign bodies infecting the organism. Like T cells in an immune system, bureaucrats often see their job as eliminating the infection or at least limiting its damage.
In a truly agile enterprise, bureaucracy and innovation become partners. They create a system where both elements improve and where people in each camp collaborate to generate superior results. We’ll show how to harmonize the two in this book.
Let’s Have You Folks Do Agile
Frederick Winslow Taylor claimed to take bureaucratic management from an art to a science. His stopwatch studies are classics in the annals of business. His 1911 book, The Principles of Scientific Management, outlined four fundamental tenets: (1) managers plan work, and workers do it; (2) managers scientifically analyze the most efficient ways for workers to work; (3) managers scientifically select and train the right workers for the right jobs; and (4) managers rigorously supervise workers as they perform tasks.4 At the time, Taylor’s methods drew harsh criticism for treating humans like machines. But his approach caught on, and indeed it has long outlived him—even today, many companies have plenty of managers and executives who are Taylorists at heart. And when Taylorists try to implement agile, bad things happen.
Here’s how it often works: Top leaders plan the agile transformation for their subordinates, not for themselves. They create a high-powered program management office to drive the change. This office generates detailed budgets, milestones, and execution road maps, complete with Gantt charts and stoplight reporting systems, to ensure conformance to plans. It creates a slew of agile teams, typically led by Taylorists who are fresh from two days of training. When one of the teams registers a success, however tenuous, the program office broadcasts it, in hopes of convincing both internal and external audiences that the initiative is working precisely as planned. Meanwhile, the leadership team continues operating much as before, supervising and (often) micromanaging their subordinates, a group that now includes members of agile teams. These leaders frequently tell the teams not only what to do but how to do it. After all, isn’t that the job of an executive?
But agile dies on the vine when it is micromanaged from above. All the agile language about self-management, testing and learning, and so forth begins to feel like a sham. Anyway, the tools of top-down management don’t work in an agile environment. Benchmarks turn out to be worthless outside their unique contexts. Predictive plans are usually wrong, because they fail to recognize or adapt to unpredictable system dynamics. We use a survey called the Bain Agility Quotient to diagnose the health and maturity of agile initiatives throughout an organization. Wherever the Taylorist approach thrives, the perception gaps between senior executives and team members are large. Senior executives describe the company’s agile initiatives as successful and satisfying. Team members, who are closer to the action, describe them as disappointing and frustrating, not much different from traditional task forces. At first we thought the executives must be lying, but we soon discovered they are merely out of touch. They are so distant from the agile work that they only know what subordinates tell them, and subordinates tell them only what they want to hear.
To be sure, some agile teams have succeeded even in Taylorist enterprises. They stay under executives’ radar, and they thrive in spite of senior management rather than because of it. But a true agile transformation requires the active involvement and support of the company’s leaders. Senior executives who really want to scale agile will do better by showing others how to do it than by sending subordinates off to training seminars. They themselves must understand agile, love it, and use its methods in teams of their own. Gandhi famously said, “If we could change ourselves, the tendencies in the world would also change.” So it is with agile.
Agile as a Quick Fix
A few companies, facing urgent strategic threats and in need of radical change, have pursued big-bang, everything-at-once agile transformations in some units. For example, in 2015 ING Netherlands anticipated rising customer demand for digital solutions and increasing incursions by new digital competitors known as fintechs. The management team decided to move aggressively. It dissolved the organizational structures of its most innovative functions, including IT development, product management, channel management, and marketing—essentially abolishing everyone’s job. Then it created small agile “squads” and required nearly 3,500 employees to reapply for 2,500 redesigned positions on those squads. About 40 percent of the people filling the positions had to learn new jobs, and all had to profoundly change their mindsets.5
Experience has revealed countless problems with this approach. It confuses and traumatizes the organization. People aren’t sure where to go or what to do. It assumes that thousands of individuals, most of whom have no experience or knowledge of agile, will suddenly understand and work according to its principles. Although radicalized converts have publicly touted their success, overall results frequently failed to meet unrealistic promises; stock prices (including ING’s) have often declined, sometimes by 30 percent or more. Behind closed doors, these executives and their subordinates are more balanced, typically offering assessments that sound something like this: “Our leaders and culture were not ready for such radical change. The more we recited conventional clichés about ‘ripping off the Band-Aid’ and ‘burning the boats’ of retreat, the more we believed them. But nobody in our senior team had ever worked in an agile environment. We did not foresee or plan for the unintended consequences. Worse yet, we lost some great people who were branded as obstructionists for trying to point out those consequences. Our approach to agility was not very agile.”
More common than big bangs in the quick-fix department is the bureaucrat’s favorite innovation tool: copying others. Of course, executives use nicer names for it—benchmarking, competitive intelligence, or becoming a so-called fast follower—but it all boils down to copying. A favorite template is Spotify, well known for its original agile lexicon of squads, tribes, guilds, and the like. Some companies even find themselves copying some other company that originally copied Spotify.
The logic of copying is seductive. Agile pioneers like Spotify have spent years learning and applying agile principles. Why not replicate that success in six months? Particularly enticing is the idea that all you have to do is copy the pioneer’s organization structure and office design. If you change the boxes and the layouts, you will surely force changes in how people approach their work. Changing the ways of working, in turn, will change outputs and outcomes. What could possibly go wrong?
Well, let us count the ways. For one, human organizations (like human bodies) are complex systems, which means that variables interact differently in different environments. Medications that work for some patients may be harmful to people with different genetics, genders, ages, or foods in their system. Managers who attempt to paste the structures of innovation departments at one company onto the entire enterprise of another are bound to produce unintended consequences. Spotify itself is sophisticated enough to understand this. It designed its engineering model to match its unique culture, capitalizing on the trust and collaboration inherent in the department’s values. Spotify’s engineering teams have fewer interdependencies than at most organizations because of its modular products and technology architecture. So would-be copiers with product lines requiring close coordination of interdependencies often end up with tribe structures that create chaos. Spotify people adamantly warn that its engineering model is constantly evolving and should not be copied by other companies or even by other areas within Spotify. Still, the copying continues.
A second problem with copying org charts: Too often, companies unintentionally destroy accountabilities in their business units. They create shiny new silos of agile teams that are every bit as challenging to integrate as functional silos ever were. General managers who once felt like CEOs of their units suddenly find themselves without the authority to make tough trade-offs. The financial performance of one company’s credit card business, for example, deteriorated significantly when key revenue and cost levers were distributed among many different tribes beyond the influence of the business unit leader. Agile teams have to support properly defined business units—units that are accountable for meaningful P&Ls. They can’t bypass or compromise those units without jeopardizing accountabilities.
Third, matrix management brings unexpected complexities. Agile teams are cross-functional teams. Cross-functional teams, by definition, require matrix organizations. Matrices may look easy on paper, but we often find ourselves cleaning up companies that launched hundreds of agile teams and never anticipated the inevitable turf battles. Who owns the teams? Who can launch additional teams? Should there be separate organization units for technology-based agile teams (sometimes called product teams) and all other innovation teams? Who funds the teams, how will decision rights work, how will teams be measured and rewarded—on and on. These details are invisible on organization charts. They are easy to overlook and impossible to copy from others.
But the worst problem is that copycats have not learned the keystone of agile success: the ability to continuously learn, evolve, improve, and grow. In trying to shortcut the process, they fail to develop skills for adapting, customizing, and harmonizing all the elements of an operating system. Agile transitions are never-ending journeys, not copy-and-paste projects. People need time to create—and get accustomed to—a new operating model. Predicting exactly how any given change will affect the organization is hard, so testing, learning, and step-by-step scaling are essential.
Agile methods, like all other management tools, have strengths and weaknesses. They do not eliminate problems. When used properly, in appropriate situations, they trade potentially disastrous problems for preferable problems. Small, autonomous agile teams are happier, faster, and more successful, but they also require more coordination and more frequent planning and funding cycles. Agile teams eliminate layers of hierarchy, but fewer layers mean fewer title changes and less frequent promotions. Failure to anticipate and address such challenges will confuse and disappoint team members. The best approach is not to choose agile over all other management approaches but to learn when, where, and how to use it in combination with other tools. This is consistent with what Aristotle, more than 2,300 years ago, called “finding the golden mean.” It is also a practical path to achieving what others have called ambidextrous organizations using contingency theory and Theory Y philosophies.
Doing Agile Right: A Road Map of This Book
Back in 2001, after software developers had practiced so-called lightweight development methods for about a decade, a group of seventeen practitioners gathered to share what they had learned about better methods. They renamed the lightweight approaches agile and created a simple set of principles to define the process. Their Manifesto for Agile Software Development helped hundreds of thousands of individual software teams adopt and apply agile practices. Today, as companies have wrestled with agile at scale for about a decade, we’re in a similar situation: we now have enough experience to analyze new patterns of success and failure. So we need to weed out the noxious misunderstandings and misuses of scaling agile before bad agile drives out good—before this powerful philosophy joins business process reengineering and quality circles on the scrap heap of management manias. It’s time to bring more sanity, practicality, and balance to the agile movement. That’s the purpose of this book. We want agile to become a valuable and practical tool rather than one more frustrating fad. We believe that agile mindsets and methods can make people in an organization far happier and more successful. We want readers to look back on their agile transitions in five to ten years with a sense of pride and fulfillment rather than disappointment and remorse.
Who will benefit from the book? We have several kinds of readers in mind. We want to help senior executives of large companies—especially large companies mired in bureaucracy—close the chasm between their bureaucratic malaise and their agile aspirations. We want to help those who are just beginning their agile journey to avoid the mistakes we have just described, and we want to help them develop agile attitudes and habitual behaviors that will create sustainable results rather than chaos. If a company has already begun its agile journey on the wrong foot, we hope to help it recognize and escape the pitfalls before it’s too late. Of course, we suspect that agile team members—and other employees who collaborate with agile teams—will also use this book to improve their performance (and maybe share it with some antiagile bosses). And we expect that start-ups already steeped in agile practices will use it to build balanced agile enterprises as they scale their success. In all these cases, our purpose is to help people build agile habits that will improve their results and increase their happiness.
For all these readers, we have tried to write a compact guidebook that time-starved businesspeople can actually read. We designed each chapter as a logical and sustainable step in the journey to an agile enterprise.
Chapter 1: How Agile Really Works. Few executives have ever watched an agile team in action. Fewer yet have actively participated in one, and almost none have actually led one. Without that kind of practical experience, leaders struggle to understand what agile is all about. In this chapter we’ll tell a detailed story that shows agile in action. We’ll explain where the philosophy came from, and we’ll outline the elements that make it such a different and more fruitful method of innovation.
Chapter 2: Scaling Agile. Scaling agile multiplies the degree of difficulty, but it can also produce exceptional results. Some organizations scale merely by adding more teams. Others pursue the goal of a truly agile enterprise, one that combines extensive use of agile teams with some bureaucratic functions, and that harmonizes the operations of both. This chapter introduces the remarkable transformation at Bosch, and it describes the steps a company must take as it creates an agile enterprise.
Chapter 3: How Agile Do You Want to Be? More agile is not always better agile. There is an optimal range of agility for every business, and for every activity within a business. But how to determine this range? Companies must find the right balance between stasis and chaos, and they must make the necessary trade-offs. They will need a new set of metrics that indicate how agile they are, how agile they want to become, whether they are moving in the right direction, and which constraints are impeding further progress. In this chapter, we’ll show you how to address these issues using agile methods.
Chapter 4: Agile Leadership. Leading an agile enterprise, as Bosch executive Henk Becker discovered, isn’t the same as leading a conventional company. Agile leaders spend less time reviewing the work of subordinates. They add value by adapting corporate strategies, leading critical agile teams, spending time with customers, mentoring individuals, and coaching teams. Changing one’s own behaviors, restructuring daily routines, and developing new skills are far more challenging than telling others that they should do so, but they are also far more valuable. This chapter shows you how to go about it.
Chapter 5: Agile Planning, Budgeting, and Reviewing. Planning, budgeting, and reviewing systems are at the heart of command-and-control environments. Dell and other agile companies don’t abolish these processes; they build agility into them. The companies conduct all three processes in frequent, adaptive cycles that rely extensively on bottom-up input. They prioritize strategic imperatives while welcoming unplanned initiatives. They regularly compare actual to expected performance to determine whether plans and budgets need changing.
Chapter 6: Agile Organization, Structures, and People Management. Companies are often tempted to copy an agile company’s organization chart, thinking that new structures will make all the difference. But it doesn’t work: you need more than structural change to break down silos and hierarchies. Agile enterprises often find themselves redesigning every element of their operating model—roles and decision rights, hiring and talent-management systems, and so on. Org charts may need to change as well. But deciding which tools to deploy, in what sequence, and to what degree requires considerable testing, learning, balancing, and customization—not copying.
Chapter 7: Agile Processes and Technology. Agile enterprises foster an obsession with customers, both internal and external. They aim to increase the quantity and quality of customer solutions. But customer solutions are only as good as the business processes that produce them, and those processes are frequently constrained by the technology that enables them. Some companies hesitate to even begin an agile transformation until their technology is ready to support it. But that can take years. Is this wise, or does it unnecessarily delay getting started?
Chapter 8: Doing Agile Right. The final chapter sums things up, proposing some rules for avoiding faddishness and describing the capabilities that have proven particularly important to success in scaling agile. We tell the inside story of Amazon, which created its own highly agile systems, tools, and ways of working, with results that have made it one of the world’s most valuable companies. And we wind up the book with a short list of indispensable guidelines for creating agile enterprises and making yourself into an agile leader.
Throughout, we hope to show how an agile business delivers measurable improvements in outcomes—not only better financial performance but also greater customer loyalty, employee engagement, and societal benefits. That, of course, is the only valid objective of an agile transformation: improving performance and better achieving the enterprise’s purpose. Agile isn’t a goal in itself; it’s a means to an end. Still, agile is about people as well as numbers. It’s about creating an organization where talented people love coming to work, and where the bars of bureaucracy’s iron cage are finally bent so that the human beings within can escape.
If you and your team are not having fun with agile, you’re not doing it right.
A Note on the Research
Stories of agile teams are fun and persuasive, probably more persuasive than they should be. The problem is that stories are so easy to manipulate to make any point a pundit wants to make. (If you don’t believe this, take a look at how differently CNN and Fox News report the exact same political stories.) Confirmation bias is a well-known challenge in digging for truth: human beings tend to look for and believe evidence that confirms what they want to hear. But is the story representative or a statistical aberration? How often has it occurred? How often have people done similar things but failed rather than succeeded?
We will use anecdotes about agile throughout this book, and we will do our best to report them fairly and insightfully. But we would like to put them in proper perspective. Agile is founded on empiricism and the scientific method. It stresses that hypotheses should be tested against real-world results rather than trusting alluring theories or intuitions. If agile works, there should be empirical data to put the anecdotes into a realistic statistical context. So, before diving further into how agile works, let’s address the more fundamental issue of whether agile works.
Before writing this book, we dug up as much research as we could find on the results of agile approaches. We examined countless anecdotes, including those from hundreds of our own clients. We examined correlations from diagnostic surveys completed by thousands of agile practitioners who track their progress using our Bain Agility Quotient. To be as objective as possible, we also collected and analyzed seventy third-party research reports. (A complete list of the reports is included in appendix C so that you can peruse any or all of them.) They include journal articles, books, government papers, academic theses, conference papers, consultancy research, corporate research, and so on. Some reports have been updated regularly for many years. Others are metastudies synthesizing the findings of several researchers. Some are more academically rigorous than others. We probably missed some reports, which no doubt will annoy their proponents. For that, we apologize and promise to continue expanding and updating our database.
Overall, we were encouraged to find so much empirical data. We find significant evidence that agile approaches at the team, scale, and enterprise levels all improve results (figure I-1). Even without controlling for the quality of agile execution or throwing out inconvenient data points, we find little research suggesting that agile approaches will, on average, hurt results. More specifically, we find the following:
More innovation improves results. If you are disappointed in your company’s performance, feeling a little unbalanced, and wondering whether to nudge the organization toward doing less or more innovation, chances are good that the right answer is more. Over 90 percent of the research reports we collected show that innovation improves business results. None show that it hurts. Apparently, few companies are doing so much innovation that it has reached the point of diminishing returns. Some research suggests that stock prices may not reflect the future benefits of innovation. But stock markets are famous for their efficiency over time, not for their short-term accuracy.
FIGURE I-1
Number of research papers found addressing each question
Agile innovation is even better than traditional innovation. We found twenty-one research reports on this topic. Three-fourths of them found that agile is a superior way to do innovation, and only 10 percent concluded that agile did not help. It is important to note that while agile methods may improve the odds of success, they do not guarantee it. One of the most popular reports, the Standish Group’s chaos study, has been comparing the success rates of agile versus traditional approaches in IT projects since 1994. These researchers now have a database of more than fifty thousand projects, and they find that agile projects are three-fifths more likely to succeed (42 percent versus 26 percent) and one-third as likely to fail (8 percent versus 21 percent). That’s impressive, but a 42 percent success rate is not 100 percent, and such percentages matter only if you are doing enough agile projects for the laws of large numbers to take hold.6
Large-scale agile teams of teams also improve results. Despite concerns that agile was designed for individual teams and can’t scale effectively, research shows otherwise. While large, complex problems have lower success rates using either traditional or agile approaches, agile’s relative advantage over traditional approaches actually increases as complexity grows.
Agile innovation works in situations beyond IT. As we noted earlier, many people believe that agile began in IT and works only there. They are wrong on both counts. Agile began outside IT but rapidly gained traction in technology as the internet took off. Fourteen of fifteen research reports find that agile works effectively in a broad range of industries and functions within those industries.
Agile enterprises can improve results. Be aware that this is a less mature area of research. There are only nine reports at this writing, and few have yet been published in the most rigorous academic journals. Still, early results are encouraging. And as we compare agile mindsets and methods to a growing body of research on business results, employee engagement, and successful leaders and teams by academics (such as Teresa Amabile, Steven Kramer, Mary Shapiro, and MIT’s Center for Collective Intelligence), consulting firms (including Gallup, Willis Towers Watson, and the Energy Project), and corporations (such as Google’s Project Aristotle), we see remarkable consistency in the findings. Collectively, this substantial and growing body of research indicates that doing agile right is likely to help executives achieve their purposes and objectives.
And now, on to understanding more about what agile is and what it takes to create an agile enterprise.
