In the early 1980s, IBM’s then-CEO John Opel declared that IBM would hit $100 billion in revenue by 1990. Although it may be hard to remember back that far, IBM was the world’s most profitable company in the 1980s. Its market capitalization accounted for roughly three-quarters of the value of the entire computer industry. Opel wanted to keep IBM from getting too complacent, so he challenged the company to increase in size from $40 billion of annual revenue in 1983 to the magic $100 billion mark by the end of the decade.

So far, so good, according to the theory of BHAGs. The problem, as it turns out, is that BHAGs can distort people’s behaviors. When the CEO of IBM speaks, he isn’t asking the organization to do something; he’s demanding. So, people will do things to satisfy the CEO even if they aren’t in the best interests of the business.

IBM built factories that would churn out the mainframes that would be needed to meet the revenue goal–even as personal computers took off and as minicomputers diminished the demand for mainframes. To try to slow demand for personal computers, IBM delayed using new chip technology from Intel–and promptly lost the PC market to Compaq and then others, who jumped on the latest from Intel. To goose revenue, IBM shifted away from a leasing model, in which IBM would book each year some fraction of the value of the machines on customer premises. Instead, IBM sold machines outright–increasing revenue in the short run but making the company much more vulnerable to short-term changes in demand. When mainframe competitors began winning customers, IBM declared that revenue was sacrosanct. Salesmen were to protect market share at all costs. Well, customers aren’t stupid. One talked about his “million-dollar coffee mug”–all he had to do was put the competitor’s coffee mug on his desk when an IBM salesman came calling, and the price of the IBM equipment would get discounted by a million dollars. IBM had ignited a price war that crushed profits.

IBM not only didn’t reach $100 billion of revenue by 1990; the company had messed itself up so completely that the company had a near-death experience in the early 1990s, and the team that had succeeded Opel was kicked out.

If you’re going to establish a BHAG, first think about all the ways it could lead to problems. How could competitors stop you? How could your own people distort your directives? (Remember, as IBM learned about its sales force, people can be very creative when trying to earn compensation.) Could some technology issue push you astray? Etc. Etc.

Proceed with humility. Remember that even Collins, as widely respected as he is, got it wrong when he singled out Motorola’s Iridium project as a laudable BHAG in “Built to Last.” Iridium didn’t just get the technology wrong, producing a satellite phone-system in the late 1990s that was comparable to the cellphone technology of the mid-1980s. Motorola and other investors also got the structure of the business wrong, when they made Iridium a stand-alone operation. Once investors turned over their money, no one had any incentive to kill the project, even as cellphone technology left the satellite operation far behind. Not Iridium management, which had options in Iridium that would be worthless if the venture was scrapped. Not Motorola, which saw Iridium as a huge customer for networking equipment. With no incentive to halt things, Iridium went ahead and spent the full $5 billion of investors’ money and opened for business in 1998, only to file for bankruptcy protection in 1999 and auction off its assets for $25 million.

In other words, it’s easy to get a BHAG wrong.

Microsoft’s experiment purports to show that “Vista is great, it is just that people are prejudiced against it.” Microsoft makes its point by bringing together 140 avowed Vista haters who had actually never used Vista. They just concluded from word-of-mouth and from reading reviews that Microsoft’s latest operating system was slow, buggy, etc. When shown a demo of “Mojave,” which they were told was a new Microsoft operating system but which was really Vista, the survey participants were enthusiastic. Thus, Microsoft’s conclusion that the problem with Vista is one of customer perception rather than the actual capabilities of the product itself.

Dan, who knows a lot about experiment design, points out that Microsoft’s experiment was fatally flawed. He acknowledges that getting people to experience Vista without preconceptions is a good idea. The problem is that the experiment provided little of the actual Vista experience, like installation and actual use without the benefit of a canned demo or an expert guide. This, Dan concludes, “transforms the experiment into a PR move and not a true experiment on the capability of Vista.”

Microsoft’s approach is not unlike the market research done by Iridium, the satellite venture that cost Motorola and its partners $5 billion to build. Iridium went bankrupt less than a year after the service was launched, and its assets were sold for $25 million. In a key piece of market research, Arthur D. Little began a survey with this statement: “There will soon be a new personal telephone service which at a reasonable cost will provide you with the capability to be reached or to place calls anywhere in the world using satellite technology, which is not limited in coverage like a cellular phone. To access the service you would have a small handset that fits in your pocket…” When asked whether they’d like such a device, respondents were quite positive, and Motorola enthusiastically reported the survey results at an investor conference. The problem with the survey question, as phrased, was that it was more marketing than market research. The responses validated the appeal of the concept but did little to validate the actual offering. For that to happen, the respondents would have had to understand that “reasonable cost” meant $3,000 per handset plus $3 per minute plus monthly fees, that “anywhere” meant anywhere with a line-of-sight view of rapidly orbiting satellites, and that “fits in your pocket” meant you needed a pocket that would hold a brick.

Motorola and other Iridium investors went on to lose billions because they believed that their market research was valid. We hope Microsoft will not confuse its marketing campaign with true market research.