When to make mistakes
Many managers recognise the value of experimentation, but they usually design experiments to confirm their initial assumptions. An advertising company typically may try different approaches to see which tactics work best but won’t run an ad that it presumes will fail. Experiments of this type aren’t deliberate mistakes. True deliberate mistakes are expected, on the basis of current assumptions, to fail and not be worth the cost of the experiment. According to conventional wisdom, they have a negative expected value. But if such a mistake unexpectedly succeeds, then it has undermined at least one current assumption (and, often, more). That is what creates opportunities for profitable learning.
When fundamental assumptions are wrong, companies can achieve success more quickly by deliberately making errors than by considering only data that support the assumptions. Research shows that executives who apply a conventional, systematic approach to solving a pattern recognition problem are often slower to find the solution than those who test their assumptions by knowingly making mistakes.
Consider the approach the Pentagon’s Defense Advanced Research Projects Agency (DARPA) has used in developing robot vehicles. Congress mandated that by 2015, one-third of US military ground vehicles be autonomous?–?unmanned and not remote controlled. Faced with that deadline, DARPA could have been expected to seek out the most experienced people and companies in the world. But it didn’t. It made the mistake of turning to amateurs.
DARPA sponsored an unmanned-vehicle race across the California desert, offering US$1 million to the winner. In 2004, the competitors barely made it off the starting line. This poor showing might seem to confirm the error of turning over such research to teams of university students. But the experience revealed the flaws in 13 different approaches, helping builders quickly zero in on designs that would succeed.
In 2005, DARPA held another race, this time offering US$2 million. Five vehicles finished the 132-mile course in Nevada, and a Volkswagen Touareg modified by a team from Stanford University took the prize. DARPA had set the stage for rapid success by deliberately encouraging a high failure rate.
Even if companies consider implementing a strategy of making deliberate mistakes, they often reject it as too costly. An oil company may know that it will get nothing from nine out of 10 wells it digs according to its best models, but drilling in locations that are assumed to be dry simply to test its model would be a very expensive proposition. Companies need to carefully analyse the trade-off between the potential expense of a mistake and the potential benefits of learning. In general, consider making intentional mistakes when:
The potential gain greatly outweighs the cost of the mistake. The expense of a failed mistake should not be too high in comparison with the potential rewards, including learning. The Bell System’s mistake cost millions and might have resulted in no improvement, but the potential payoff?–?reducing a US$450 million bad debt?–?was very large. David Ogilvy’s cost of running additional ads was fairly small, considering the potential benefit of learning about what worked.
Naturally, companies need to limit risks in their mistake making and avoid moves that could be catastrophic. For a maker of jet engines, testing experimental equipment that’s expected to fail would be foolish in an aircraft full of passengers. Testing it in a simulator or wind tunnel would be smart.
Decisions are made repeatedly. A strategy of knowingly making errors is likely to be valuable in environments where core assumptions drive large numbers of routine decisions, such as those about hiring, running ads, devising promotional tactics, or assessing credit risks.
Giving credit cards to college students was a radical idea when it was proposed at Citibank in the 1980s, and many cognoscenti saw it as a mistake. Immature consumers with high debts, no jobs or homes, and limited or non-existent credit histories were considered the worst possible risks.
But Citibank issued the cards anyway, without requiring adult co-signers, and discovered through this mistake that parents would often bail out their card-holding children when they got into trouble and that many of the students turned into valuable long-term customers.
Companies frequently rely on rigid models (such as assuming students are bad credit risks) in their routine decisions. If a deliberate mistake can help an organisation develop and apply better models, the benefits will be multiplied over a large number of future decisions.
Conversely, for a decision that a company expects to make rarely or only once, such as relocating headquarters, committing a deliberate mistake makes little sense.
The environment has dramatically changed. Companies need to make missteps to fully appreciate the extent of significant changes in the competitive landscape.
In a new environment, organisations are very likely to make inadvertent mistakes anyway, because current approaches may no longer work. Deliberate errors can help companies learn even faster. As they say at Procter & Gamble?–?which operates in a market where very few product introductions succeed –?“fail often, fast, and cheap”. With old product-development models becoming less effective, P&G famously challenged its assumption that innovations should come from inside the company and turned to outside partners for new products such as the SpinBrush electric toothbrush. In an environment that is changing quickly, the strategic advantage shifts to those who learn fastest?–?and rapid learning may require deliberate mistakes.
The problem is complex and solutions are numerous. The more complex an environment becomes, the less likely you are to understand it completely. Intentional mistakes can help you see the weaknesses of your mental models and explore other ways of approaching a problem.
In the entertainment business, the rise of cable, TiVo, the internet, the iPod, and other technologies has created such a complex environment that experiments, and mistakes, are becoming ever more important. Some of the most successful entertainment innovations, from 24-hour news channels to reality television to immediate releases of downloadable TV shows, challenged long-held assumptions.
Your organisation’s experience with a problem is limited. If you’re unfamiliar with a problem, you should be open-minded when approaching it. Suppose the company is expanding into a new market.
Executives will be tempted to apply models and strategies borrowed from the old market. But the organisation may not be able to assess for some time how well those models fit the new situation. Making deliberate mistakes at the outset can help expedite learning that may be required for the company to establish a beachhead.
In the 1980s, most large grocery chains had little experience with the growing market for organic foods. What if they had made the error of setting up large sections of their stores, or even independent chains, to focus on this segment? Whole Foods Market did essentially that when it opened its first small store in Austin, Texas, in 1980.
Today, Whole Foods has more than 180 stores in North America and the UK, with fiscal 2005 sales of US$4.7 billion, and most traditional supermarkets are now expanding their organic food sections. Could the traditional companies have learned faster by making a mistake in this arena before Whole Foods became a dominant player?
Which mistakes to make
Of course, there are many mistakes you could make on purpose, from hiring inexperienced people to see if they’ll succeed to jumping off a bridge to see if you’ll get hurt.
How do you distinguish between smart mistakes and dumb ones? Our management consulting firm, Decision Strategies International, has developed a step-by-step process for identifying and executing smart mistakes, and we use it ourselves to practice a strategy of deliberate mistake making. In one early test, we assumed a mistake we were planning to make would lose a significant amount of money, but the opposite happened.
By turning major assumptions on their heads, our company created more than US$1 million in new business. Using the context of our own firm’s mistake making, we’ll explain the steps we use:
Identify assumptions. The firm’s leaders first identified many deeply held assumptions about how best to run the business. These axioms became the raw material with which we intended to make mistakes that could reveal faulty assumptions. Here are 10 of the assumptions we identified:
1 Cold calling Fortune 100 prospects does not work.
2 Our clients buy primarily on trust and reputation, with limited price sensitivity.
3 Young MBAs don’t work well for us; we need experienced consultants on the team.
4 Bundled pricing is better than separate pricing for each of the project’s components.
5 Senior partners must get more pay from their billing bonuses than from their base salaries.
6 Formal interviews with clients must always be done by two consultants, with one taking notes.
7 The firm can be successfully run by a president who is not a senior consultant with significant billings.
8 Executive education and consulting are natural cross-selling activities.
9 Books and articles are vital to the firm’s image as cutting-edge and rigorous.
10 Responding to requests for proposals is not worthwhile, because organisations that send them out are usually price shopping or are just going through the motions to justify a choice that’s already been made.
Top executives may not be aware of all the assumptions a company makes, so they should ask colleagues throughout the organisation to help construct such a list. Focus on assumptions that lie at the core of the business in areas including strategy, operations, marketing, finance, legal matters, IT, and human resources.
Select assumptions for testing. The firm’s management team ranked the assumptions according to their importance to the enterprise and how confident the team was about their accuracy.
To assess importance, the executives asked themselves what they would do differently if they knew the assumption was false. The more differently they would behave, the more important the assumption was considered to be. To assess accuracy, they asked themselves what they would be willing to bet that the assumption was correct: The company? Their reputations? Their careers?
The firm chose to test three assumptions –?3, 7, and 10?–?that were deemed very important and highly accurate. (Assumptions people disagree on are likely to be tested in the normal course of business and thus may not be good candidates for a deliberate-mistake strategy.)
Rank the assumptions. To score the relative value of testing each assumption, four company managers were asked whether they agreed with each of a series of statements that map directly to the five conditions, listed previously, under which companies should consider making deliberate mistakes. The statements were:
“The potential gain from a mistake in this area greatly outweighs its cost.”
“We rely on this assumption repeatedly.”
“The business conditions surrounding this issue have changed.”
“This is a complex problem.”
“Our experience with this assumption is limited.”
They scored each statement on a scale of 1 to 7, with 1 meaning they agreed “not at all” and 7 meaning they agreed “completely”. Assumption 10, which holds that responding to RFPs is not worthwhile, had the highest score and was thus selected as the most likely to benefit from a strategy of intentional mistakes.
Create strategies for making mistakes.
The firm decided to make the mistake of responding to the next RFP, which was from a regional electric utility. To keep costs down, our management team assigned recent hires to develop the proposal. Nevertheless, the RFP was taken very seriously and the proposal was developed with great care.
Execute the mistake. Our firm produced an elaborate, customised response that listed the partners’ normal fees, resulting in a total budget of more than US$200,000 for this relatively small consulting project. To our surprise, the electric utility invited our firm to visit the CEO and the senior management team to explore not only the RFP but other projects as well. After learning more about our firm’s capabilities, the utility company signed a separate project on short notice before accepting the original proposal. More work followed, amounting to more than US$1 million of additional consulting from this client.
Learn from the process. Mistakes cost time and money. Whether the outcome is good or bad, conduct a careful analysis afterward to understand what you have learned. How does the result reinforce or change your assumptions? What were the surprises? How might the outcome change your business? What other experiments or mistakes might be useful?
This relatively low-cost deliberate mistake has changed our firm’s thinking about RFPs and about mistake making in general. We have responded to several RFPs that we would have ignored before. As part of a systematic process of making mistakes, the firm is also testing other cherished assumptions, including 3 and 7 on our list. Hiring young MBAs is obviously easier than replacing the person running the firm, so we have hired some junior staff. And, as a low-cost approach to testing assumption 7, we are experimenting with a shift in the president’s balance between consulting and management.
Many wrongs can make a right
The wisdom of profiting from mistakes, deliberate or otherwise, is not lost on savvy executives. In bringing Craig Mundie, who had founded a supercomputer company that ultimately failed, to Microsoft, Bill Gates noted that “every company needs people who have made mistakes? and then made the most of them.”
Unfortunately, the people who most need to make mistakes are the ones least likely to admit it, and the same is true of companies. Overconfident individuals and businesses are usually uninterested in subjecting their presumed expertise to empirical tests. But to be more successful in the long run, managers sometimes need to be less successful in the short run. Leaders who understand this will foster an organisational culture that encourages intelligent, deliberate mistake making.
As IBM’s Thomas J. Watson Sr. noted, “If you want to succeed, double your failure rate.”
Harvard Business Review
Join the CIO New Zealand group on LinkedIn. The group is open to CIOs, IT Directors, COOs, CTOs and senior IT managers.