The article answers it's own question. On the website developed for Obamacare:
> The government had spent $400 million over four years - more time than it took the U.S. to enter and win World War II - and yet, the dozens of contractors couldn’t set up a website to take sign-ups.
So the answer to the headline question (and also the broader problem) is: because the government probably can't develop the project successfully in-house.
This is not a particularly American problem. Every few years I read about some super costly government IT disaster here in the Netherlands. I'm sure locals from most other countries will have similar stories to share. So the broader question would be: what makes governments apparently unable to get big IT projects done right? (I'm aware that there is a big selection/reporting bias in the disaster stories.)
According to the article, Booz Allen got a 10 year contract. What would it take, when it expires, for the BLM to develop and run this successfully in-house?
- Government jobs don't pay well relative to the private sector, especially in software, so the government talent pool is lower quality than industry.
- For most government activities, government essentially has a monopoly on their implementation, e.g. infrastructure building. So people don't have as much of a comparison to know how good or badly government is doing. With software, people use government software and use privately developed software and the difference is obvious.
- Software, perhaps even more than many endeavors, benefits from having a single decision-maker in charge of the project who has a clear vision of what needs to be done and strong conviction on how to do it. Design-by-committee is not a great way to design good software. Most government projects wind up being some form of design-by-committee, either implicitly or explicitly. Even when the government has software built by private contractors, the requirements are written by committee, the contractors chosen by committee, etc. Very few of the best of anything is designed by committee, but the effects are much more obvious with software than say, a park.
Healthcare.gov was farmed out to well-connected contractors, much like most other similar boondoggles (SLS, F-34, Bradley fighting vehicle, delayed road projects, etc).
It says a lot more about corrupt and inefficient procurement (e.g. cost-plus) than it does about rank-and-file government employees. Using many contractors in many different congressional districts, all of whom have little incentive to deliver on time and on budget, or cooperate effectively with each other.
Fundamentally you're talking about the problem I alluded to in the 3rd point: Software projects work best with a single (or small group of closely knit) decision-makers at the top who bear ultimate responsibility for the implementation of the project.
The problem with governmental design-by-committee is that it spreads out the responsibility such that the failure of the project does not fall on any individual or small group.
The lack of concentrated responsibility breeds an environment that incentivizes grift rather than delivering value.
If the iPhone bombed, it was going to be Steve Jobs' fault. He had final say in all design decisions. If Healthcare.gov bombed... whose head rolled? Excerpt from the wiki article of Todd Park, CTO in the Obama administration:
'''
The initial version of HealthCare.gov, which was deployed on July 1, 2010, was built in 90 days by Park and his team at HHS. The first HealthCare.gov was cited by the Kaiser Family Foundation as one of the early highlights in the implementation of the healthcare reform implementation progress. HealthCare.gov was also the first website ever "demoed" by a sitting president
The following two versions, from the relaunch of the front end in May 2013 to the badly flawed marketplace that went live in October 2013, were developed by contractors and overseen by officials at the Centers for Medicare and Medicaid Services, outside of his purview within the White House Office of Science and Technology Policy. When the extent of the problems with Healthcare.gov became clear, Park was tasked by President Obama to work on a "trauma team" that addressed the "technological disaster". Park, along with Jeffrey Zients, led the "tech surge" that ultimately repaired Healthcare.gov over the winter, eventually fixing the marketplace sufficiently to enable millions of Americans to find plans and purchase health insurance.
'''
How many of those unnamed officials at the Centers for Medicare and Medicaid services got fired for a multi-billion dollar boondoggle? The reason bureaucracies (both government and private) love process, documentation, committees, etc., is because the primary goal of any bureaucracy, far and above its nominal mission, is to continue existing, and a primary part of continuing to exist is to avoid blame for anything. Delegating decision making to committees and process is a key way to avoid blame.
"Who is responsible for this mess?"
"Nobody is responsible, we followed the process. But don't worry, we've started the process of forming a committee to update our process manual to prevent this mess in the future."
Is your claim here that corporations are able to correctly assign responsibility for major failures? Why do you think this? There are so many ways to deflect blame for your failures onto others while taking full credit for successes you had nothing to do with in the private sector.
Not exactly -- my claim is the the most successful organizations (both governmental and private) have high alignment between power and responsibility. This produces good incentives.
Bureaucracies (both governmental and private) tend towards poor alignment for the reasons I outlined.
Most mature companies eventually become bureaucracies, a process I like to refer to as "becoming IBM". Big tech companies (e.g. Google, Amazon) are well on their way to becoming IBM, and much of the dysfunction is exactly due to your point about deflecting blame and taking credit for successes.
So the distinction is not between government vs private, but rather bureaucracy vs non-bureaucracy. Many governments, such as the US, do tend to be highly bureaucratic -- much more so than e.g. IBM.
I feel it still says something about government employees because one can pretty much take it as a given that the standard employee pool is basically incapable of building relatively basic e-commerce websites.
Well yeah, what do you expect when Congress and the GSA sets pay for pay for developers at levels far below the private sector? I've worked for two different federal agencies in the past, and both were more efficient and lower BS than most private-sector jobs I've worked at. This wasn't IT or development, of course.
It's just taken as a truism that government is always inefficient and bureaucratic, but the average ossified corporation such as Google is wildly inefficient and bureaucratic just the same.
They also aren't necessarily that bad to interact with. I'd much rather go to the DMV than interact with Comcast.
The standard government employee is a project manager whose main job is overseeing contractor work. Funding cycles make it nearly impossible to hire in house staff to actually do work, and there is firm political opposition to the federal government doing anything on its own.
I've known some people who did contract work for the DoD like how you describe. They reported to DoD employees and were otherwise indistinguishable from actual DoD employes. However, there was some company that did their W2s, and took a cut almost as large as their salaries.
Being someone who just happened into a government job for my first one out of college that holds true in my experience. No good employee stayed in the job more than two years and most only one before they got so sick of the politics and bike shedding.
The government gets good employees it just loses them all very quickly when it fails to compensate them as well as provide them with meaningful work.
Another thing to consider is that the fundamental goal of most software is to automate tasks that are done by humans. Having this development take place inside the organization already doing those tasks is going to be met with hostility, and will likely cause projects to settle into non-optimal solutions that typically look identical to the clunky workflows they are replacing, but with an electronic medium of exchange rather than a paper one.
>- Government jobs don't pay well relative to the private sector, especially in software, so the government talent pool is lower quality than industry.
If we're discussing the USA, sure. If we're discussing other countries this isn't always necessarily true, and as the person you responded to has noted, this happens worldwide.
That is just one of the three points I made (and I personally think the final point is the most salient here). Though I'm curious, do you know of any countries where government software jobs pay market rate? I would be interested to see if their government websites are decent.
I think government could offer software engineers the same thing it offers others: not the best pay, but stable employment with a good pension package and a predictable workload. Not for everyone, but it could certainly appeal to some people.
* According to the Standish Group’s Annual CHAOS 2020 report, 66% of technology projects (based on the analysis of 50,000 projects globally) end in partial or total failure. While larger projects are more prone to encountering challenges or failing altogether, even the smallest software projects fail one in ten times. Large projects are successful less than 10% of the time.
* Standish also found that 31% of US IT projects were canceled outright and the performance of 53% ‘was so worrying that they were challenged.’
* Research from McKinsey in 2020 found that 17% of large IT projects go so badly, they threaten the very existence of the company.
* BCG (2020) estimated that 70% of digital transformation efforts fall short of meeting targets. A 2020 CISQ report found the total cost of unsuccessful development projects among US firms is an estimated $260B, while the total cost of operational failures caused by poor quality software is estimated at $1.56 trillion.
I'm in the consulting world and I've been brought in at the tail end of multiple BILLION dollar boondoggle modernization efforts. These are fortune 100 companies that built their fortunes through acquisition and consolidation who have no idea how to steer their ship in any sort of effective way. In a lot of cases, we're cleaning up after other top 5 consulting firms who lead the client on while pissing away hundreds of millions of dollars. It turns out damn near everyone is bad at building large systems but only the government seems to be derided for it consistently.
I think there's a difference between "this product doesn't make money" and "this product does not work". Amazon's alexa is a decent product that is a failure because it doesn't make money. Most failed private IT projects are this. The government isn't trying to make money. Their failed products are in the "this product does not work" category.
Big IT projects fail. The US federal government for the most part only deals in big projects. There's more to it than that, but they're mostly details compared to this basic truth.
> So the broader question would be: what makes governments apparently unable to get big IT projects done right?
Signals intelligence people must be building some of the most powerful computer systems in the world. They also historically have innovated and led the industry in areas like crypto. How come they get it right but the rest of the civil service can’t?
The rest of the civil service seems almost entirely unable to get anything done in computing though. Signals intelligence in the UK and US seem to ship quite a lot of success, from all the history, leaks, and things you can physically see like Bumblehive. Definitely something they’re doing differently than the rest of the civil service.
> The government had spent $400 million over four years - more time than it took the U.S. to enter and win World War II - and yet, the dozens of contractors couldn’t set up a website to take sign-ups.
So the answer to the headline question (and also the broader problem) is: because the government probably can't develop the project successfully in-house.
This is not a particularly American problem. Every few years I read about some super costly government IT disaster here in the Netherlands. I'm sure locals from most other countries will have similar stories to share. So the broader question would be: what makes governments apparently unable to get big IT projects done right? (I'm aware that there is a big selection/reporting bias in the disaster stories.)
According to the article, Booz Allen got a 10 year contract. What would it take, when it expires, for the BLM to develop and run this successfully in-house?