It is not enough for a Project Manager to be satisfied that the project has been well managed and delivered on time, on budget. Many, if not most, Project Managers wrongly believe that is the sole measure of their success.
A project is not successful unless it delivers
positive benefit to the organisation.
It is not fully successful unless it delivers optimum benefit.
What is benefit?
How do we define it?
How do we measure it?
There is a tradition, particularly with finance managers and accountants, that benefit is simply a question of projecting the effect on profit (ie revenue minus expenditure) over a given period of time and comparing it against the position the organisation would be in without the project, ie:
|revenue if the project succeeds||minus||costs of the project||minus||operational costs with the project||)|
|(||revenue if there is no project||minus||operational costs if there is no project||)|
This form of calculation is normally referred to as Cost/Benefit Analysis (CBA). Note that the formula works even where there is no revenue concerned - just a cost reduction.
One way of stating the result is in terms of the "pay back period", for example, it might take three years before the equation gives a positive result. A project paying back in six months sounds better than one paying back only after three years.
More complicated versions of the cost/benefit calculation may be made to take into account the timing of the cash flow. If I pay out £10M now and get back £11M in two years time, I am probably losing money - not gaining it. I could have invested that £10M and made more from it than the additional £1M over the two years. In any case, inflation probably means that the £11M is not worth more than the £10M was two years earlier. This form of financial analysis is normally called "Discounted Cash Flow". The results are often presented in terms of "present day value" - for example, the £11M in two years may be equivalent to having £9M today.
The project may be treated as a form of investment. For example, over a period of five years this project will result in a 15% return on the investment. The question for the organisation is then - is that the best use of our money or could we invest it in a better initiative or investment opportunity? Organisations often apply a standard benchmark "Internal Rate of Return" which they consider represents how much return they would normally be able to achieve on their funds. If the project exceeds that figure it generates positive net benefit and is to be supported. If not, the money could better be spent elsewhere. IRR is used to calculate present day value in a discounted cash flow calculation.
These forms of financial analysis tend to require specialist accounting input. Unless you are familiar with finance, you should probably refer the financial case to the organisation's financial management.
For many types of cost and benefit you will be to calculate a projected value. For example, you can estimate:
(Accounting for the time of project participants is considered further in the section on accounting.)
Even in these cases, there will often be difficult assumptions and estimates to make. It is even harder to estimate things like:
One possible source would be to refer to industry benchmark information. You may be able to show, for example,
...therefore a good eSolution would reduce invoicing costs by 90%.
Benchmarking information can be very valuable, but there are some issues to bear in mind:
The biggest drawback in taking a purely financial perspective is that it can be impossible to put a firm value on many components of the value. A classic example is the benefit of "better management information" which is stated as a benefit in most project proposals.
Here is an all too common cost justification approach:
|Cost of project||£10M|
|Direct cost savings from project||£2M|
|Better Management Information leading to a 1% improvement in revenue and cost control||£9M|
|Net Benefit||£2M + £9M - £10M = £1M|
The real problem is often that you cannot cost justify the project unless you put a price on such things. Who said there would be a 1% improvement and how did they know? The usual answer is that their guess was as good as anybody's so no one could challenge the analysis. The reality is that creative accounting can justify almost any initiative - if there is the will to do so.
One technique for putting a value on things which have no direct valuation is to ask the business leaders how much they would be willing to pay to achieve the non-financial benefit. If you can get the executive to say that they would be willing to pay £9M to have good management information, that would be another way of putting a price on its value.
Finding the financial justification for eProjects at this time can be even harder. Many eCommerce solutions, particularly business-to-consumer eSolutions, are not aimed at immediate financial gain rather than establishing market share in the belief that market share now will lead to profits in the future. Such speculation is very hard to capture in a purely financial business case. The benefit of the eProject is more properly stated in terms of its true intentions such as lower overhead costs, improved customer service, wider catalogue, opening up new markets, etc.
An even harder benefit to define for an eProject, is the belief that a business has to enter the eCommerce arena if it wishes to survive at a time when significant market share is being diverted into eCommerce channels. What price do you place on survival? If we take the net value of the organisation as a factor in the cost justification we can probably justify almost anything!
A purely financial perspective of benefits has its limitations:
|It is wise, therefore, to consider all forms of benefit that could be anticipated from the project, whether financial or non-financial, measurable or unquantifiable.||Financial||Non-financial|
A good technique for this is to borrow the "Balanced Business Scorecard" approach, first expounded by Robert Kaplan and David Norton in the Harvard Business Review, January 1992. The scorecard is a technique to direct attention at all areas in which good performance is required to build a successful business.
In addition to the obvious financial bottom line, it focuses on the customer's perspective, the internal organisational perspective, and the efficiency of the business processes. The key point is that these other perspectives are the things you have to get right in a business to generate financial success. So, why measure financial results when you could be measuring the things that create the result - much earlier in the chain.
|Customer Perspective||"Balanced Business Scorecard"||Organisational Learning Perspective|
||Internal Process Perspective|
When looking at the benefits from a project, examine all potential types of benefit, whether or not they are financial, whether or not they are measurable, and whether or not they formed part of the sponsors' initial expectations. It is always good to exceed the sponsor's expectations by identifying additional benefits.
Where possible, find a way of quantifying the benefit. This will be used to demonstrate that a benefit is being achieved even if it is not a direct measure of the benefit. For example, you can show that your project will (or has) saved three days in the average sales cycle or that monthly reports are available five days earlier. You do not need to know the financial impact of those improvements to be able to discuss them as benefits. In particular, you can state them in terms of:
Almost all forms of benefit can be tracked somehow, although you may need to find innovative ways of measuring them, for example, staff satisfaction could be tracked by recording the average time taken as sick leave or staff turnover rates. If your task is to improve the efficiency of a call centre, this could be a very significant thing to measure.
This overall statement of all types of expected benefit is known as a Benefit Model. It is used throughout the project to measure success.
The most important time to consider the anticipated benefits is during project start-up:
As part of the Project Definition work:
These definitions of anticipated benefits form an important management tool for assessing the success of the project (and, therefore, the success of the Project Manager). They must be fully understood and agreed by the project sponsors and other relevant parties. In particular, the targets must be achievable and agreed if they are to be used as performance measures for the project or for the Project Manager and other participants. It should be made clear whether targets represent anticipated performance or are "stretch targets" - ie goals to be striven for but which probably surpass reasonable expectations.
Information from the benefit model, particularly the projected costs, will be used to set up the project's detailed budget.
In those cases where the project is to be undertaken by a third party organisation under commercial terms, benefit targets often form part of the contract and can be the subject of stage payments, contingency/performance-related payments and/or penalty clauses. Clearly, great attention needs to be paid in these cases and appropriate specialist legal advice should be taken.
Benefit tracking should be a frequent or continuous process during the project. The Project Manager's prime goal should be to deliver optimum benefit. As each new phase of work commences:
There are three main uses for benefit tracking during the project:
The basic principal behind performance measurement is that you get what you measure. If you tell someone that they are being measured on specified results they will understand that those things are important and they will attempt to excel for their own personal rewards (whether tangible or just in terms of personal satisfaction). One word of caution, some people excel against measured criteria at the expensive of unmeasured ones. Be careful how you state any targets. For example, if you say speed is the key benefit, do not be surprised if the designers remove all the control processes.
Benefit measurements can also be used in decision making - particularly for change requests. An obvious factor in accepting a change is whether doing so will increase the overall anticipated benefit. Conversely, the impact of an issue may have been considered in terms of its negative effect on benefits. Any significant change in the anticipated benefits will need to be reported and discussed at the relevant management level.
Project Managers can use the current benefit predictions as one form of executive reporting. It is possible to construct simple "dashboard" style indicators to show current status or the trend to date. For example, you could construct the current cost/benefit calculation once a month, based on the latest completion and resourcing estimates, to show graphically how expectations have been improving - or not. Where the Project Manager is calculating and reporting "earned value" and "estimates to complete" using automated project management tools, it should be reasonably simple to prepare regular update reports on the project's projected financial benefit.
You might also show non-financial benefits as a trend, for example, how the projected time to fulfil a customer order has changed during the lifetime of the project and against the original target. Clearly, though, the project manager would need access to detailed information about the designs and developments that the project team are doing to be able to calculate and report such things. But then, isn't it the Project Manager's job to know what's happening and how well the project will match up to expectations???
Ideally, benefit tracking should be a continuous management process. In the real world, many project managers cannot afford the time for frequent checking and reporting of all types of benefit. Indeed, in many projects the Project Manager has much more pressing tasks to perform and would not be thanked for spending time analysing and reporting when there were fires to put out.
One time that it always makes sense to review the anticipated benefit is at phase end when we assess the success of the work to date and make plans for the next steps. In an extreme case, it might be apparent that the project in its current form is not going to produce optimum benefit. With the rapid pace of technology, any slow project runs the risk of being out of date before it is halfway through. It is always wise to repeat those sanity checks from the Project Definition. Will this project deliver benefit? Will it deliver optimum benefit?
Without doubt, the project's sponsors and management team should review the anticipated benefits achieved at the end of the project. Lessons can be learnt. Further improvements can be planned and instigated.
Projects are usually defined to finish shortly after the live implementation of the business solution. At that time the realised benefit equation is looking at its worst - all the costs have been incurred and none of the benefit has been realised. It should be possible to review the anticipated benefits - but not the actual ones.
Once the live solution has settled down, it is best practice to hold a Post-Implementation Review (PIR). Its purpose is:
The benefits that were identified, itemised and quantified for the project should be visible throughout the operation of the live business solution. If they were considered important benefits for the project they probably also form important performance criteria for the management of that live operation. The line management should be encouraged to continue monitoring performance against the benefit targets. The continuing focus on delivering the benefit may be used: