Category Archives: Government Contracting

Spend more on delivery, less on risk mitigation

Let’s do a simple Lean analysis of government IT system delivery projects. How much of our spend is on activities that directly create value, and how much is additional overhead? What percentage of our spend is value-creating?

The value-creating part of a software development project is primarily the actual development and testing of the software. Add to that the cost of the infrastructure on which it is run, the cost of designing and building that infrastructure, and perhaps the cost of any software components from which it is built. I mean to include in these costs the salaries of everyone who is a hands-on contributor to those activities.

The non-direct-value-creating part is primarily management overhead and risk mitigation activities. Add to these the costs of the contracting process, documentation, and a bunch of other activities. Let’s call these overhead. A great deal of this overhead is for risk mitigation – oversight to make sure the project is under control; management to ensure that developers are doing a good job; contract terms to protect the government against non-performance.

No one would claim that these overhead categories are bad things to spend money on. The real question is what a reasonable ratio would be between the two. Let’s try a few scenarios here. An overhead:value ratio of 1:1 would mean that for every $10 we spend creating our product, we are spending an additional $10 to make sure the original $10 was well-spent. Sounds wrong. How about 3:1? For every $10 we spend, we spend $30 to make sure it is well spent? Unfortunately – admittedly without much concrete evidence to base it on – I think 3:1 is actually pretty close to the truth.

Why would the ratio be so lopsided? One reason is that we tend to outsource most of the value-add work. The government’s role is management overhead and the transactional costs of contracting. Management overhead is duplicative: the contractor manages the project and charges the government for it, and the government also provides program management. Another reason is the many layers of oversight and the diverse stakeholders involved. Oversight has a cost, as does all the documentation and risk mitigation activity that is tied to it. When something goes wrong, our tendency is to add more overhead to future projects.

A thought exercise. Let’s start with the amount we are currently spending on value-creating activity, and $0 for overhead. Now let’s add incremental dollars. For each marginal dollar, let’s decide whether it should be spent on overhead or on additional value creation (that is, programmers and testers). Clearly we will get benefit from directing some of those marginal dollars to overhead. But very soon we will start facing a difficult choice: investing in more programmers will allow us to produce more. Isn’t that better than adding more management or oversight?

To produce better results, we need to maintain a strong focus on the value creating activities – delivery, delivery, delivery.

Who needs requirements?

On my other blog, I posted an entry on how agile approaches in a way dispense with the idea of requirements; instead a business need is translated directly into code (skipping the requirements step), with tests providing an objective way to see whether the result is acceptable.

This idea disturbs many government IT and procurement professionals. It shouldn’t.

Perhaps it will ease people’s minds to think of an agile process as something like a procurement done with a Statement of Objectives. In place of system requirements the government, throughout the course of development, presents the contractor with business needs, and the contractor is free to provide a solution without constraints. For the same reason that this is often good practice in contracting, it is also good practice in software development. I am not saying that agile procurements should be done through a Statement of Objectives (a good idea in some cases), just pointing out the underlying similarity in concept.

One objection I hear is that without requirements, we cannot contract for services. Even if we could, how could we have a fair competition, since contractors bid on how they would address requirements? The trick here, I believe, is to distinguish between contractual requirements and system requirements. There is no rule that says that the contract or the RFP must include system requirements. Of course it must include some sort of requirements. The requirements depend on the basis for the competition – for example, if a procurement is for development services, we can state requirements for the services – required skills and experience, management approach, etc. Or we can state requirements for the business needs to be fulfilled. Perhaps the following comparison is in order: if I wanted security guard services I could specify that the security guards need to prevent people we don’t trust from entering the building. The solicitation does not need to list the names of the particular people we don’t trust.

A second objection is that we need the requirements to know whether the contractor or the project team has performed well. That seems to miss the point. If the requirements are satisfied but the product doesn’t meet the business need, then no one has been successful. We should gauge success by business value produced, business needs met, quality of work, customer service, and so on. Or we can judge the contractor’s success at meeting the business needs developed in the “conversations” with users. We don’t need system requirements in the solicitation to do this.

The main point to keep in mind is that better results are obtained by working from business needs directly to system development. Best results are what we want. We might have to change how we set up our contracts to get there. There is no conflict, from what I can see, with the Federal Acquisition Rules.

Good technical practices are critical for government contracting

Good technical practices (such as those typical in DevOps environments) can help the government in contracting for information technology services. We should require these technical practices in our IT services contracts, and if we are investing in QA and independent verification, we should invest first on validating good technical practices. Let me give a few examples. For readers without a technical background, you should be able to find more information about these practices online. 

Good, state-of-the-art testing practices are important for more than the obvious reasons. Most tests should be automated and should follow the classic “testing pyramid” (many unit tests, somewhat fewer integration tests, and fewer tests at the user interface level). The automated tests themselves are just as important a deliverable from the contractor as the code itself.

There are many reasons why such automated tests are important in our contracting environment. The automated tests serve as regression tests that will speed later work on the system. If a second contractor does something that “breaks” the first contractor’s code, it will immediately be spotted; in essence, the tests can be said to “protect” the first contractor’s code. If a new contractor is brought in for O&M or future development, the automated tests serve as documentation of the requirements and allow the new contractor to be confident in making changes or refactoring – they are OK as long as the regression tests continue to pass. 

Scripted deployments and “infrastructure as code” serve a similar function. By providing automated scripts to set up the production environment and deploy code, the contractor is documenting the deployment process (and reducing the amount and cost of paper documentation!). No longer is the knowledge just in their heads (making it costly to replace the contractor). Deployment scripts can be tested, making them an even more valuable form of documentation. They can be placed under version control and audited, increasing security.

Continuous integration increases our ability to work with multiple contractors and gives us more confidence in a contractor’s status reports. By continuously integrating code we ensure that code from multiple contractors will interoperate, and we avoid last-minute surprises when a contractor’s 100% finished work fails to integrate.

A zero-defect mentality where user stories are tested immediately and defects are remediated immediately ensures that code the contractor says is finished really is finished. It avoids passing defective code from one contractor to another; reduces finger-pointing; and makes integrating code simpler. If we are comparing contractor performance it serves as an equalizer – if one contractor finishes 10 stories and leaves 15 defects while another contractor finishes 8 similarly sized stories and leaves only 12 defects, which has performed better? We can’t know. Zero known defects should be our expectation.

The last practice I will mention is the use of good design patterns and architectures that feature loose coupling. Good use of design patterns makes it easier for a new contractor to understand the code they inherit. By encapsulating pieces of the system it can make it easier to have multiple contractors work in parallel and even at different paces.

Together, these practices can make it easier to judge contractor performance, allow us to partition work between a number of contractors, and make it easy to switch contractors over time.

(thanks to Robert Read at 18F for some of these ideas)