Chapter 14

Measuring Project Performance

In This Chapter

arrow Understanding how performance will be measured for each project.

arrow Working out whether you’re delivering your project on time and on budget

arrow Establishing whether you are on track for future successful project delivery or not

KPIs are incredibly useful strategic and operational navigation tools. They allow you to know where you are against expected targets and make real time adjustments to ensure you end up where you want to end up. Often, at an operational level, the execution of strategy is facilitated by the implementation of a number of projects.

It is therefore essential to measure project performance so you know you are going to hit your targets and are fully across the operational implementation of the strategic objectives.

The KPIs detailed in this chapter offer up practical insights into project efficiency and effectiveness.

Why Project Performance Matters

Projects matter because most strategic and change initiatives are delivered via projects. Project performance matters because it’s essential to monitor these projects carefully to make sure they deliver the objectives they were initiated to deliver.

Keeping an eye on project performance allows you to assess current performance levels, provide input into future goal setting and decision making, and help anticipate any potential problems. If you don’t monitor project performance then you are likely to run into difficulties. There are many ways to screw up a project and there are plenty famous examples to focus your mind on the importance of measuring project performance. For example:

  • The new Wembley stadium: The Football Association’s centrepiece was originally scheduled to open in 2003 but it didn’t open until 2007.
  • Sydney Opera House: Australia’s architectural icon was scheduled to open in 1963 at a cost of $7 million but actually opened in 1973 at a cost of $102 million.
  • Concorde: The supersonic airliner cost 12 times more than scheduled.
  • The Channel Tunnel: The link between the UK and France cost 80 per cent more than budget
  • Boston’s ‘Big Dig’ tunnel: This huge construction project went 275 per cent – or $11 billion – over budget.

Clearly the bigger the project the more you need to pay attention. Project failures and over runs whether time or budget often carry severe consequences – and not just financial. For example, the software giant Oracle has been sued for an alleged $20 million budget overrun of one of their software implementation projects.

remember.eps If a project is important enough to be initiated it’s important enough to measure.

Introducing the Three Components of Project Performance

Projects usually have a defined beginning and end as well as a defined budget. There purpose is to bring about a change or deliver an outcome that is outside ‘business as usual’ or normal business operations.

There are always three key components of project performance and delivery against all three will determine success. The three components are:

  • Schedule: Is the project on schedule?
  • Budget: Is the project on budget?
  • Deliverables: Is the project delivering the specified outcomes?

If you want to measure the success of your projects and ensure they deliver what you intend them to deliver then you must monitor and measure performance against schedule, budget and quality of output.

Tracking whether your projects are on time (Project Schedule Variance)

The project timeline is usually a critical component to any project. The purpose of the project is to bring about a result, improve an output or deliver an outcome by a specified day. That day may be determined by a myriad of different and sometimes competing factors. It may be that you are hosting an event and you need a venue built by a certain date. There is no movement on that completion date because the event is already locked in. It may be that you need to complete a software development project which is just part of a bigger project and the hand-over must happen on an exact day, or the software will not be completed for the client on time. In other instances the timing is not as critical and the project may have a delivery window of several days. Generally speaking the quicker you want a project completed, the more expensive it will be.

Project Schedule Variance is a simple comparison that allows you to measure the planned or scheduled project time and the actual time taken to complete the project.

If Project Schedule Variance is zero then the project was completed on time. If the variance is negative it shows an overrun. And if the variance is positive it indicates that the project was completed ahead of schedule.

For larger, longer projects it’s always wise to monitor progress as you go along rather than just waiting until the completion date to see what ‘falls out’. Measuring progress and milestones throughout the project allows you to compare where you thought you’d be at a certain point against where you actually are. That way you can adapt and fine tune to get back on track as soon as possible.

Measuring whether your projects are on budget (Project Cost Variance)

All projects have a budget – or should have – to provide strict financial parameters. It is possible to achieve just about anything in just about any time frame if you have deep enough pockets, but few commercial organisations have the luxury of a limitless budget.

The decision to execute the project in the first place is usually based on detailed expectations around what the project will deliver. Ideally, you should have analysed the project to ensure there is sufficient return on investment to warrant going ahead. If, however, the budget that analysis was based on is subsequently blown, then the return on investment disappears. Measuring whether your projects are on budget is therefore incredibly important throughout the life of the project, to ensure that it delivers what it promises without spending more than you have allocated.

remember.eps Project Cost Variance is a simple comparison of the planned or scheduled project costs and the actual costs to complete the project.

If Project Cost Variance is zero then the project was finished on budget. If the variance is negative then you sent more on the project than you budgeted for. And if the variance is positive you finished the project with money left in the kitty.

For larger, longer projects it’s always wise to monitor budget as you go along rather than just waiting until the end. Measure progress and anticipated spend as you progress so that you can compare what you thought you’d have spent at a certain point against what you actually spent. That way you can adapt and fine tune to get back on budget as soon as possible.

Checking whether your projects are delivering the right value (Earned Value)

Earned Value (EV) is a particularly useful metric for helping you to monitor progress throughout the project, so you can assess whether your projects are delivering the intended value.

EV is a project tracking measure that looks at the cost of work in progress and allows you to understand how much work has been completed compared to how much was expected to be completed at any given point.

In addition to assessing progress to date, the EV metric allows you to project what the likely costs of the complete project will be; assuming performance levels remain as they have been to date. This can be very useful in anticipating over spend before it happens so evasive measures can be taken to minimise the overspend where necessary.

Measuring the KPIs in practice

The most useful project performance KPIs to measure are Project Schedule Variance (PSV), Project Cost Variance (PCV) and Earned Value (EV).

KPI: Project Schedule Variance

The key performance question project schedule variance helps to answer is: ‘To what extent are our projects delivered on schedule?’ The data for this KPI usually comes from a project management software application or via manual records. PSV is usually measured monthly but can be monitored more often for important short-term projects.

Project Schedule Variance (PSV) = Scheduled Completion Time (SCT) – Actual Completion Time (ACT)

ACT and SCT are measured in time intervals such as days or weeks.

For example say you are running three separate projects:

  • Project A: SCT = 67 ACT = 90
  • Project B: SCT = 23 ACT = 19
  • Project C: SCT = 56 ACT = 57

The PSV for each is as follows:

  • PSV Project A = 67 – 90 = –23
  • PSV Project B = 23 – 19 = 4
  • PSV Project C = 56 – 57 = –1

This KPI can also be adapted to measure performance across a business or department. For example if you want to calculate the departmental Project Schedule Variance you simply add the individual project variances for each separate project together for an actual number or calculate a straight or weighted average variance score.

PSV Department = (–23) + (4) + (–1) = –20

remember.eps Your target is zero. Ideally you should be aiming to deliver your projects on time every time. Overruns usually come with negative cost and implications for your reputation.

For the most part zero is also better than a positive PSV. If one of your managers consistently delivers ahead of schedule then it is actually likely to indicate weak planning, with too much room and contingency built into the plan. Aim for just-in-time delivery.

KPI: Project Cost Variance

The key performance question project cost variance helps to answer is: ‘To what extent are our projects delivered on budget?’ The data for this KPI will also usually come from a project management software application, financial planning applications or manual records. PCV is usually measured monthly but should be measured more frequently for important projects.

Project Cost Variance (PCV) = Scheduled Project Cost (SPC) – Actual Project Cost (APC)

For example going back to the three separate projects:

  • Project A: SPC = $400,000 APC = $500,000
  • Project B: SPC = $350,000 APC = $325,000
  • Project C: SPC = $200,000 APC = $300,000
  • PCV Project A = $400,000 – $500,000 = –$100,000
  • PCV Project B = $350,000 – $325,000 = $25,000
  • PCV Project C = $200,000 – $300,000 = –$100,000

To calculate the overall Project Cost Variance simple add the individual project variances for every separate project together for an actual number or calculate a straight or weighted average variance score.

PCV Department = (–100,000) + (25,000) + (–$100,000) = –$175,000

Like the PSV, your target is zero. Ideally, you should be aiming to deliver your projects on budget every time. Overruns can cause financial problems. A positive PCV is, of course, usually a good thing. It means you didn’t spend as much as you thought you would. But it could indicate poor planning, and the ‘extra’ money set aside for that project could have been better used elsewhere.

tip.eps For longer, larger projects break the costs down into milestones throughout the project to help you see how you are tracking as the project moves forward. That way you can avoid any nasty surprises at the very end.

KPI: Earned Value (EV)

The key performance question earned value helps to answer is: ‘To what extent are our projects delivering the intended value?’ Again the data you need to calculate EV will usually come from a project management software application or any manual project records. How often you measure EV will depend on the project type but weekly or monthly are common. EV is always specific to an explicit point in time (or status day) you select.

In order to calculate EV you require three values:

  • The overall budgeted costs for the project: This is usually referred to as the ‘Budgeted Costs of Work Scheduled’ or BCWS for short and includes any costs (resource costs and fixed costs) associated with the delivery of the project. You can calculate BCWS for any point in time (status day) along the project delivery timeframe simply by multiplying it by the percentage completion of a project at that given time. For example, if a project is scheduled for 4 months with an overall price tag (BCWS) of $100,000, then the BCWS is $25,000 after month one, $50,000 after month two, and so on. After the first month 25 per cent of the project have been completed, leading to the following calculation: $100,000 × 0.25 = $25,000. For month two it would be $100,000 × 0.5 = $50,000.
  • The actual costs used up to run the project up to the status day: This is usually referred to as ‘Actual Costs of Work Performed’, or ACWP for short.
  • The value of work delivered at the given project status day: This is the value that has been earned by the project work and is often referred to as ‘Budgeted Cost of Work Performed’, or BCWP for short. For example, if after 2 months of the project 75 per cent of the work on the project has been completed, then you might expect that also 75 per cent of the total project budget have been spent ($75,000 for our example here).

You can now use these figures to assess project performance. Let’s go back to our example of a $100,000 project scheduled over 4 months.

Earned Value (EV) = Budgeted Cost of Work Scheduled (BCWS) × % complete

The chosen status date could be after 3 months of running the project, at which point 60 per cent of the value of work has been delivered. Or in other words, the project is 60 per cent complete. The earned value, or (BCWP) is $60,000. However, the scheduled value (BCWS) is 75 per cent at this 3 months milestone. This tells us that the project is not on schedule because less value has been earned compared to what was planned.

Now let’s look at the actual costs up to that 3 month status date. If we find that $90,000 (because for example a more expensive contractor had to be hired in) it tells us that the project is not only delivering less value than expected but is also over budget.

You can also calculate the factor of overspend or underspend, called Performance Level:

Performance Level = Actual Cost of Work Performed (ACWP)/EV

So for our 3 month status date, we have:

Performance Level = $90,000/$75,000 = 1.2

If we take the assumption that the same level of overspend (or underspend) continues for the rest of the project, then we can use these figure to calculate a predicted forecast:

Predicted Forecast = (Remaining cost × Performance Level) + ACWP

So in this case it would be:

Predicted Forecast = ($25,000 × 1.2) + $90,000 = $120,000

This means that, based on current progress and spending the project will overrun by $20,000.

warning.eps EV is easily misinterpreted. Although the rhetoric surrounding EV states that it can deliver an objective quantitative measure of project performance in reality its validity hinges on how project progress is reported.

Like all KPIs it’s important not to get too hung up on the final number. Instead use EV as a tool to identify and rectify potential problems in the project delivery. EV relies on good reporting and evaluation, so it’s important to be aware of the potential for cheating!

There are several ways to manipulate EV. For example:

  • Deliberately scheduling projects for longer than necessary so the driver of the project can complete more quickly and look good.
  • Deliberately inflating the budget so the driver of the project can look good.
  • Deliberately load all the easy tasks at the start of the project so good progress is made initially, keeping a project EV score looking good for a long time.
  • Deliberately exaggerate the task completion percentages.

These types of unwanted behaviours can easily and quickly skew the data and render EV meaningless.