What We Learn from Case Studies

businessmen talking

We studied many of the most visible project failures of the last decade including Merck’s Vioxx, Denver airport baggage handling, Columbia Shuttle, New York City police communication, Airbus A380 wiring problem, Boston’s Big Dig ceiling tile collapse, and the failure of both the Mars Orbiter and Lander. Then we asked what lessons could be learned from these failures.

Four lessons stood out. They don’t explain everything, but they can be useful in generalizing what can go wrong across a broad spectrum of projects.

Here is a summary of some of what we learned.

Overconfidence

There is a tendency in some projects like the NASA Columbia Shuttle, and Denver’s baggage handling project to suffer from overconfidence at exactly the wrong time. At NASA, engineering management was confident that breakaway foam posed no threat because it had occurred on every flight before the February, 3 2003 disaster. They were wrong, and seven astronauts perished. At the beginning of the Denver airport project, the design team was warned that baggage handling, given the scale of the airport, was extremely complex and that the proposed design was pushing the envelope in automated baggage handling. Yet, airport authorities, together with United, its major tenant, continued for over a decade to try to make th3e system work. They never did, and the system was abandoned in 2005.

Is overconfidence generally a problem? In their classic book, Decision Traps, Russo and Shoemacher present compelling evidence that most decision makers are overconfident across a wide range of decisions. Project managements are no exception and can be overconfident in their ability to meet budgets, schedules and project objectives. There are, of course, stages in the project where confidence in the team’s ability to effectively execute plans is critical. The challenge to project managers is to know when it is appropriate to be confident and when not to let overconfident behavior jeopardize the success of the project

Ignoring New Evidence

The second lesson we learned was that managers too often ignore new evidence, especially when that evidence suggests that the project is on the wrong track or that project strategy needs to be revisited. This was especially evident in Merck’s Vioxx, the NASA’s Challenger, and Microsoft’s Xbox. Merck had evidence at several stages that the drug could lead to heart attacks and strokes, yet the project was never halted. As a result, up to 20,000 people may have suffered from taking this medication. In NASA’s Challenger there was previous evidence that the “O” ring could lead to problems during a cold weather launch. And this is exactly what happened. Seven astronauts died. The Presidential Commission investigating the disaster pointed to a “bureaucracy that overlooked a faulty seal design and overruled dire warnings against launching the shuttle.” At Microsoft, customers complained about the high failure rate of the Xbox game system, which they called the “Red Ring of Death.” Their complaints, which started shortly after the product was launched in late 2005, were ignored by the company until the problem became so large that the company, in July 2007, was forced to increase the warranty period from one to three years, and repair the units at a reported cost of $100 for every unit sold. The total cost to Microsoft for ignoring the problem was estimated in excess of $1 billion.

Knowing when new evidence needs to be used to re-evaluate project strategies proves to be a critical factor in project success.

Unrecoverable Costs

The third lesson was that money already spent cannot be recovered and should not be considered a factor in continuing the project. A project, at any time over its life cycle, has value only if its objectives can be met. When it becomes clear that the outcome is doubtful serious consideration must be given to abandon the project. Or, at the very least, the project needs to go back to the drawing board. The Denver baggage handling project, as an example, was in trouble after its original debut, but management couldn’t let it go. They persisted in trying to solve, what many believed to be, an unsolvable problem. Time after time they chose to throw good money after at a bad project. Merck also suffered from the same problem and was unwilling to abandon Vioxx after spending $5 billion on clinical trials and FDA approval.

Maintaining commitment to a losing cause is not unusual behavior. It happens when investors are reluctant to sell a stock as the price falls. It happens when angels invest more money to keep a poorly reviewed Broadway show open for just a few more weeks, and it happens when managers find it difficult to terminate employees who repeatedly fail to meet quotas. In short, most people are hesitant to pull the plug on any kind of failure.

In short, past investment should not be a consideration when determining the fate of project. The only reason to continue is that the project is likely to be successful. Sounds obvious doesn’t it?

Priorities

Every project has many objectives. It must be completed within budget, on schedule, and meet quality objectives. But there is another important strategic objective that must be met; the project must be successful in helping the firm meet its competitive objectives. We can call this a market objective. If the project meets its budget, schedule, and quality objectives, but fails in the marketplace because a competitor introduces a much better product, has the project been successful? Did the company attach a high enough priority to market success?

In almost every project not all objectives are given equal priority. At the beginning of a project, quality and market objectives may dominate the discussion. Next budgets and schedules dominate. Indeed budgets and schedules may continue to receive the highest priority over the life of the project. This may be expressed in an emphasis on Scope statements, Work Breakdown Schedules, Activity Schedules and Gantt Charts.

In our study we found that projects that place to much emphasis on budgets and schedules had the greatest likelihood of failing. Not that these projects ignored quality and market objectives, but there was evidence that they were of secondary importance. Consider several examples. When the Columbia Accident Investigation Board submitted their review of the shuttle disaster they said, that schedules and costs, under a project philosophy of Better, Faster and Cheaper, had dominated safety (quality) issues and that the safety culture of NASA had deteriorated over the years. In the case of Vioxx the priority was placed on profitability and markets, but not on quality. For Xbox the priority was again on profits and markets while quality was apparently shortchanged. The failures of the Mars Orbiter and Lander are additional examples of how budget and schedule pressures contributed to the failure of these projects. Again quality was sacrificed.

Why do quality and market suffer? Perhaps it is because budget and schedules are easily quantified and as the accountants say, “You get what you measure.” Perhaps it is because the dominant focus of project management methodology is on cost and schedule issues. Perhaps it is because quality and market competition can be ambiguous concepts.

Regardless of why the emphasis is placed on budgets and schedules, two steps need to be taken. The first is to ensure that priorities are assigned and the appropriate tradeoffs in selecting more of one than the other are carefully determined. The second is that priorities change. They change because markets are not static. New entrants break into a market, new products appear, and consumer tastes change. For this and many other reasons related to budgets, schedules, and quality considerations, priorities need to be reconsidered periodically. To take the position that NASA took … Better, Faster, Cheaper … is to invite disaster.

Do Companies Learn From These Failures?

Some do and some don’t. Merck now ‘rewards’ project managers who put an end to shaky projects. Pfizer, learning form Merck’s Vioxx debacle, made the decision in December 2006 to end drug trials of Torcetrapib, the company’s most promising drug to treat heart disease. It took this action because eighty-two people taking the drug died versus fifty-one in the same trial who had not taken the drug.

Some organizations have trouble learning. Consider that the Columbia disaster followed some 17 years after the Challenger disaster, and consider that both investigation boards pointed to management problems. In this case it is not a stretch to conclude that NASA had a learning disability!