Entries Tagged ‘Iterations’

Does your embedded development team’s project budget metric support your estimation process?

Wednesday, December 15th, 2010 by Robert Cravotta

As an engineering project lead I had to develop and report on a set of performance metrics that we called the VSP (vision support plan). The idea behind these metrics was to show how each area of the company was directly supporting the company vision statement. For many of the metrics, the exercise was a waste of time because there was no clean way to measure how what we were doing as a team directly corresponded to every abstract idea in the vision statement.

However, there were a few metrics that we used that I thought were useful because we could use them to experiment with our processes and measure whether there was an improvement or not. For example, I refused to use a budget metric that only focused on whether we came in under budget or not. My budget metrics were “green” (good) if the expenditures to date were within 10% of the budget. If the project was more than 10% higher or lower than the budget, I reported the project as yellow. If the project was more than 20% higher or lower than the budget, I reported the project as red.

Here was my reasoning for the grading. If the project was within 10% of the budget, we were in control of the budget. I believe that any team can affect the cost of a project by up to 10% by choosing appropriate trade-offs without adversely sacrificing the quality of the project. Any design trade-offs that are made to affect a 10 to 20% change from the plan involve more risk and might adversely affect the quality of the project. Likewise, any time a team must accommodate changes that stray more than 20% from the plan involve significant risk and may require a reevaluation to determine whether the project is scoped realistically.

Note that this metric specified a range that covered overruns and underflows of the expenditures to budget. A major reason for this was to put a special focus on how well we were estimating projects. How many times have you seen someone try to explain why their project is over budget? In general, the reasons I saw included one or more of:

1) There was additional scope added to the project that you did not capture additional budget for (often at the direction of management).

2) The project involved solving some unexpected problems and there was not enough (or no) budget to handle such contingencies.

3) Management would not accept a realistic budget number for the project and you are doing the best you can with the budget they offered you.

The thing that is common in all of these reasons is that the estimation process did not adequately capture the project’s predictable and iterative costs. Too many times I would see management strip out our contingency budget which usually consisted of specifying 1 or 2 design iterations at those points of the design where we had the most risk. Capturing a budget metric and putting it into the context of how good the estimate was provides the potential for finding clues as to how to improve the estimating process in future projects – which directly supports just about any company’s vision statement that I have ever seen.

Likewise, if your project was substantially under budget, it seemed that most management was content to leave that alone; however, I see the following scenarios as reasons to why you might be running under budget:

1) You overestimated the cost to perform the project

2) You were able to remove scope from the project that was left in the budget numbers

3) You made an innovative leap that increased your productivity beyond what you thought you could do during the budgeting process.

Each of these reasons had a profoundly different impact on how you refine your estimating process. The first reason suggests you need better estimators. The second reason suggests you need to improve your project and contract management process. The third reason is one that any manager should want to see more of and reward the team for making it happen.

I saw many project estimates game the system so that the project lead had a potentially oversized surplus in their budget and their management would fail to comment on how resources had been allocated to a project and then not used without uncovering which of those three scenarios was the cause for the under run.

Does your project budget process enable you to improve your estimating process, contract management process, and increase the chances that your team will gain recognition when a risk pays off when you discover a new and better way to solve a problem? What are other ways you use expense/budget metrics to improve your design team’s performance?