Government oversight bodies take great pride in canceling failing projects. They consider it a measure of success. What are oversight bodies for? Eliminating wasteful investments, of course. At first glance this might seem to be consistent with an agile mindset. Among the advantages of an agile approach are the transparency given to stakeholders and the ability to manage risk by working in increments. Only the current increment is at risk, since the project can be stopped before the next increment begins. Problems are exposed through transparency and oversight can take advantage of the incremental approach to stop a failing project.
This way of thinking is a mistake. The project was started because of a mission need. Canceling the project leaves that mission need unmet. Oversight has failed in two senses: (1) it failed to make the project successful, and (2) it did not allow a mission need to be met, one that was important enough to have invested in. If, in fact, the mission need is important, then a new project will have to be started to address that same need. The new project will have the overhead of starting up a new program, thereby wasting more money. Instead of canceling the program, the oversight body should shift the course of the current program to make it more successful.
But isn’t that throwing more money into a wasteful program? No – what has been spent so far is a sunk cost, and there may be some salvageable assets. Doesn’t the program’s failure to date mean that it is poorly managed? Not necessarily, but if it is, the oversight body should simply force the program management to change, not cancel the program. In many cases the problem is not management, but circumstances outside their control. The oversight body should help the program overcome those outside forces. Terminating the program is just a way to punish the program’s management, and adds waste for the government as a whole.
Why cancel a successful program? If a program has delivered substantial value to date, then the oversight body should consider whether the remaining work of the program is necessary. If the program’s work was appropriately prioritized, then there should be diminishing returns to continuing the program. Oversight should constantly reassess the value of the remaining work, and see if the agency’s needs have changed in a way that the remaining work is no longer worth the investment. If the oversight body decides that this is the case, it should cancel the remainder of the program and rejoice – for this it can legitimately claim an oversight success!
Agile delivery approaches focus on maximizing business value rather than blindly adhering to pre-determined schedule and scope milestones. On the definition of “business value” the agile literature is appropriately vague, for business value is defined differently in different types of organizations. I would even argue that it is necessarily different in every organization – each company, for example, is trying to build a unique competitive advantage, and results that contribute to that advantage can be valuable (“net” value, of course, would have to consider other factors as well). A publicly held company needs to maximize shareholder value; a closely-held private company values … well, whatever the owners value. A nonprofit values mission accomplishment. What does the government value and how does it measure value?
The answer is not obvious. Mission accomplishment is certainly valued. But different agencies have different missions and for some agencies measuring mission accomplishment is difficult (James Q. Wilson’s book Bureaucracy is great reading on the topic of agency missions). If the Department of Homeland Security values keeping Americans safe, how can it measure how many Americans were not killed because of its actions? In an agile software development project, how can we weigh cost against that sort of negative value to determine which features are important to build?
To make matters more complicated, the government values many things besides mission accomplishment. Controlling costs, obviously. Transparency to the public and to oversight bodies. Implementation of social or economic goals (small business preferences, veterans preferences, etc.). Auditability – evidence that projects are following policies. Fairness to any business that wants to bid on a project. Security, which in the government IT context can extend to keeping the entire country safe. And through appointed political agency leadership, political goals can also be a source of value. Each of these values may add cost and effort to a project.
To maximize business value, we must consider all of these sources of value. If we limit ourselves to the value of particular features of our software, we are missing the point. Rather, as IT organizations in the government, we need to self-organize to deliver the most value possible, given all of these sources of value. The government context determines what is valuable. What we must do is find the leanest, most effective way to deliver this value. This is no different from the commercial sector – only the values are different.
The US government is, deliberately and structurally, a low trust environment. Think about why we have a “system of checks and balances.” We have proudly created a government structure that is self-correcting and that incarnates our distrust of each branch of the government. Why is freedom of the press such an important value to us? Because we all want transparency into the government’s actions – not to celebrate its fine management practices, but to know when it is doing something wrong. Within the government, we have Inspectors General to investigate misbehavior, Ombudsmen to make sure we are serving the public, and a Government Accountability Office. To work in the government is to work in an environment where people are watching to make sure you do the right thing. It is a culture of mistrust.
That sounds horrible, and from the standpoint of classic agile software development thinking, it is unworkable. But take a step back – don’t we sort of like this about the government? “Distrust” has unpleasant connotations, but as a systematic way of setting up a government, there is a lot to be said for it. It is another way of saying that the government is accountable to the people. You could almost say – you might want to hold on to your chairs here, agile thinkers – that mistrust is actually a value in the government context. So where does that leave us if agile thinking wants us to deliver as much value as possible, but believes that agile approaches require trust?
It might sound academic, but I think solving this dilemma is critical to finding ways to bring agile thinking into the federal government. A typical IT project experiences this structural distrust over and over: in the reams of documentation it is required to produce, in the layers of oversight and reviews it must face, and in the constraints imposed on it.
I will argue that even in a low trust environment, agile approaches are still the best way to deliver IT systems. And that certain tools – borrowed primarily from DevOps – actually help us resolve the dilemma. Waterfall approaches fit well with mistrustful environments by holding out the promise of accountability and control – but they just don’t work. So how can we bring agile, lean, team-based processes into an environment that is structurally mistrustful, and realize our goal of a lean bureaucracy?