Sort by
Blog in Context: Big IT Projects Fail All the Time in the Private Sector

David Callahan

Are the technical problems of more evidence that big government can't do anything right? That's the claim of many critics of Obamacare. In fact, though, problems with complex information and software systems are extremely common, both in the public and private sector. 

Last summer, for example, one of the largest electronic trading firms in the world -- Knight Capital Group -- introduced a new trading systen, the Retail Liquidity Program. The day the system was launched, it malfunctioned because of a software bug, leading Knight Capital to lose $440 million in 30 minutes. Knight Capital was so badly wounded by the disaster that the firm's stock sank 75 percent and it ended up being folded into another firm. 

Post-mortems of the Knight debacle blamed poor software development and testing models. Sound familiar? It sure does to experts who study the failure of large software projects. 

Since 1994, the Standish Group, an IT firm, has published something called the CHAOS Report, which tracks the success and failure of large IT projects worldwide. Its 2012 report, analyzing data from 2010, found that just 37 percent of all IT projects studied could be categorized as successes. A full fifth of the projects were clear failures. And 42 percent were "challenged" -- as in struggling to succeed, which is where is right now. 

Senator Marco Rubio might have considered these grim statistics before he blithely said on Sunday that “In the 21st century, setting up a Web site where people can go on and buy something is not that complicated.”

In fact, is far more complex than many of the private IT systems that fail every year. As the Times recently reported

According to one specialist, the Web site contains about 500 million lines of software code. By comparison, a large bank’s computer system is typically about one-fifth that size. . . . 


A part of the system, hidden from users, draws data from several federal and state databases to determine if consumers qualify for coverage and then calculates the subsidies for which they may be eligible. Another part of the system sends enrollment data to insurers. Several people involved in the project say that problems like those of the last three weeks are not uncommon when software from several companies is combined into a large, complex system.

None of this is an excuse for the failures that led to the debacle. But that failure needs to put in context. The lesson here is not that government is incompetent. It's that large IT systems are devilishly difficult to get right, and is a more complex system than most.