Chapter 5

Developing a KPI

In This Chapter

Identifying the critical questions that need answered

Identifying the right KPIs for your needs

Assessing data collection techniques

The vast majority of executives fully acknowledge the need to measure performance in real time, so they can work out if they are on track toward their objectives or not. However, they don’t always know how to develop the right KPIs. How do you distil the mountains of data into meaningful action-orientated information that can direct decision making? How do you find the necessary needles in the ever-expanding haystacks?

This chapter helps you to answer those questions so you can develop the right KPIs for your business needs.

My working life is spent helping organisations define and articulate their strategy, develop dashboards or scorecards around that strategy and developing the right KPIs that will help them to monitor and manage their business successfully and deliver those strategic objectives. What I find time and time again is that any prior effort to achieve this result has been thwarted by a back-to-front approach to the process. Too often organisations fall into the trap of retro-fitting objectives to existing and established measurement protocols. An objective is not necessarily a good objective just because you can already measure it!

KPI development must start, not finish with strategy.

The Question is The Answer: Developing Key Performance Questions (KPQs)

The nature of KPIs is to provide answers. But answers to what?

There is no point wasting time and energy sourcing answers to questions you didn’t ask or couldn’t care less about. To ensure that you don’t I developed a concept called Key Performance Questions (KPQs). When it comes to developing KPIs the question is actually more important than the answer – at least it is at the start of the KPI development process. You need to know what questions you need answers to before you develop your KPIs. And those questions are KPQs. Essentially a KPQ is a management question that captures exactly what you need to know when it comes to each of your strategic objectives.

The rationale for KPQs is that they trigger a search for meaningful answers and focus your attention on what actually matters – what you need to discuss to improve performance. More importantly, they provide guidance for choosing the right performance indicators. For example, you may be able to work out the average age of your customer but does that information help you achieve your objectives? If it doesn’t, you don’t need to know it!

Business is tough enough without making it tougher with unnecessary KPIs.

Harnessing the power of questions

The main reason for strategic performance management is to improve future performance. Improvement in anything depends on learning. Real learning is only possible when you take the time to reflect on past results and reflection is facilitated by questions. Just think about it for a moment: When I ask you a question it triggers a search mechanism in your brain. This is the start of a thinking and reflection process which constitutes the beginning of learning. KPQs are therefore essential components of good performance management because they help to put the data into context and turn it into actionable knowledge (see Figure 5-1).

KPQs help you to identify your information needs and ask yourself: ‘What is the best data and management information we need to collect to help us answer our key performance questions?’ As a result the KPQs ensure that, by default, all your subsequently designed performance indicators are relevant to your business.

Figure 5-1: Relationship between KPIs, KPQs and Learning

KPQs help you and your executive team to:

• See the wood from the trees regarding what’s important and what’s not.
• Identify the most important unanswered questions in your business.
• Understand the relevance of the data sought because KPQs indicate to everyone what your company’s biggest concerns are.
• Open communication and guide discussion.
• Make better evidence-based decisions.

KPQs help everyone to appreciate and stay focused on the key concerns of the business.

Google, one of the most successful and admired companies on the planet run their company by questions. In considering their strategic performance management process, Google have formulated about 30 KPQs that they need to answer in order to ensure that they stay successful. Google executives recognise that asking questions stimulates conversation and debate, and that innovation emerges from this dialogue. Innovation is essential for many companies, especially companies like Google, yet they recognise that innovation is not something that they can just add to a bucket list of strategic objectives. Innovation is something that they need to facilitate, and questions make that possible.

Creating good key performance questions (KPQs)

There is a right and wrong way to go about creating your KPQs. For a start you need to get comfortable with asking questions! As children we ask questions constantly, irritating our parents with endless curiosity and ‘But why’s’. Then we grow up and we stop because we don’t want to look like we don’t know the answer. After a few years in business we can almost be afraid of questions because the pressure to have all the answers is so acute. And yet businesses that facilitate a very open, questioning culture always out-perform those that don’t.

There are ten steps to ensure that you create good KPQs:

• Identify one to three high level KPQs for each strategic objective on your strategy map.
• Make sure that your KPQs are performance related.
• Engage your colleagues in the creation of the KPQs.
• Make your KPQs clear, short and punchy.
• Phrase your KPQs as open questions.
• Make sure you KPQs focus on the present and future.
• Seek to refine and improve your KPQs over time.
• Use your KPQs to guide your KPIs so they deliver relevant and meaningful information that answers your KPQs.
• Use your KPQs to challenge and where necessary refine your existing performance indicators.
• Include your KPQs in the reports you communicate within the business to review performance.

A few years ago a large blue chip company approached me to audit their performance management approach. As part of their strategy, they had moved to a business model that focused on building and maintaining close partnerships with their suppliers and so it was clearly important to manage those partnerships effectively. In their drive to assess the health of their partnerships, they designed a generic questionnaire which they outsourced to a data collection agency, who reported back with detailed assessments – including graphs, charts and trend analysis – on the questions from the survey. When I spoke to the partners, however, it was clear they were less than thrilled by the survey. It contained about 50 questions that took about three days to complete every six months. Ironically the partners were happy with the relationship but irritated by the data collection!

When I pressed my client about what they were using the information for, it transpired that all of the data they were collecting was ‘interesting to know’, but that they hadn’t made a single decision based on the information collected from those surveys in over three years. They were creating unnecessary work for themselves, and more importantly their partners, for no discernible benefit – not to mention the cost of conducting the survey and getting the results.

Together we went back to the drawing board to really ascertain the key performance questions to which they were actually seeking answers via the survey. And it turned out there was only one: ‘How well are our partnerships progressing?’ With that as the target we were then able to design a system that automatically e-mailed a very simple form to the account managers with just two questions: ‘How would you assess the relationship with our company?’ and ‘How well is the partnership progressing?’ In response to the first question recipients could choose from three options: problematic, indifferent and positive. The second again offered three options: Worse than before, same as before, better than before. There was also space to write additional comments if the recipient wanted to. The simplified survey was sent out monthly because it was much easier for the recipients to complete, and more importantly it flagged potential issues early enough so they could be rectified before they escalated into an big problem. The company now has a very simple monthly data collection system in place, which allows them to get all the information they need to answer their KPQ. It saves them time, money and effort while giving them the real time information they need to manage the partnership relationship effectively, improve performance and they’ve stopped irritating their partners with lengthy unnecessary surveys. That’s the power of KPQs.

If you are ever tempted to collect and distribute information in your organisation first ask yourself, ‘What is the KPQ I am trying to answer with this data?’ If the data does answer relevant questions then send it out and make sure you also circulate the KPQs to help put the information in context for the recipient.

---

Examples of KPQs

Some powerful KPQs might be:

• To what degree are our customers likely to recommend us to others?
• To what extent are we growing profit margins among our new customers?
• How well do we facilitate innovation in our culture?
• To what extent are we raising customer lifetime value?
• How engaged are our employees?
• To what extent do our project teams trust each other?
• How well are we communicating our strategy internally?
• To what extent are we improving customer loyalty in segment X?
• How well are we promoting our products and services in China?
• To what extent are we growing market share in the South West region?

---

Deciding on the Right KPIs

Your ideal KPIs will consist of a customised suite of indicators that deliver exactly the information you need but no more. That means KPIs that help you and your team to make better decisions, improve performance and guide strategy.

In order to help you decide on the right KPIs for your business I have designed a ten-step performance indicator decision framework. This framework ensures that you apply a rigorous thought process to every potential KPI that you identify or design. Make sure you work through this ten-step process for every new KPI you want to add to your regular measurement agenda, and apply the process retrospectively to any existing KPIs that your business currently uses.

I’ll explain the process below and then provide an example so you can appreciate its usefulness in action.

Step 1: Linking KPIs to strategic objectives

Ask yourself, ‘Does this KPI link to our strategic objectives?’

KPIs need to be clearly and directly linked to the strategic objectives of your business, division or project i.e. what matters the most. When KPIs are tightly linked to objectives they provide relevant information that can then be used to monitor progress and stay on track.

Don’t make the mistake of assuming that it’s impossible to measure some of the more intangible or complex aspects of strategy. Everything can be measured – as long as you get really clear about exactly what you want to measure.

Step 2: Identifying the unanswered questions

Ask yourself, ‘Does this KPI help to answer specific, important unanswered questions that will help our business?’

The only reason you should include a new KPI or keep an existing KPI is if it helps to answer important and relevant questions that are currently unanswered. And this is where the KPQs come in. Take a step back and identify a key performance question (KPQ) you want answered.

The KPQ further defines the context of the KPI and narrows the area of interest still further from a ‘big picture’ strategic objective to a much more specific key question. The KPQ puts a manageable boundary around exactly what information is needed and why. If you can’t identify the KPQ you are seeking an answer to then it’s unlikely you need the indicator.

Step 3: Isolating the decisions to take

Ask yourself, ‘Will this KPI enable us to make specific decisions better?’

Once you have identified a pertinent KPQ, there are still likely to be several KPIs to choose from. In order to further filter the KPIs so you can chose the right one for your needs, seek to isolate the decisions you could take as a result of the KPI information.

By articulating the KPQ and the possible decisions a potential indicator will help to address, you can reduce the shortlist still further. Again, if you’ll make no decisions as a result of the KPI then you probably shouldn’t collect it. Discard that KPI and find another more suitable one.

Step 4: Checking for existing data and methods

Ask yourself, ‘What existing data and data collection methods already exist in the business?

Assuming your potential KPI has passed the test of the first three steps then you need to check for existing data and data collection methods to ensure you don’t waste time designing something that already exists or could exist very easily.

It may be that someone in a different part of your business is already measuring the KPI you are interested in. Or perhaps someone has already used your proposed data collection method. Alternatively, a simple Internet search could reveal a variety of potential collection methods that you could easily adapt rather than starting from scratch.

Step 5: Collecting meaningful data in time

Ask yourself, ‘Can we collect meaningful data of the right quality, at the right time for this KPI?’

Assuming you can identify a pertinent data collection method, you need to ensure that you can actually collect meaningful data when it’s needed. You need to make sure that your source data is available, that it’s in the right format, is of the right quality and that it’s possible to collect the data at the required frequency.

If you discover that you can’t collect meaningful data for the KPI then you need to go back to step four and identify another way to collect the data that will be feasible.

Step 6: Assessing the usefulness to answering the question

Ask yourself again, ‘Does this data help us answer the KPQ?’

Even if you can collect meaningful data for your KPI, and data can be collected in the right format, quality and frequency it’s important to double check against the KPQ. Is this potential KPI still actually helping to answer the key performance questions?

Once you start this ten-step KPI decision framework it’s easy to shift your focus from identification to design. Make sure you double check that the KPI and data collection method you are assessing still help you to answer your KPQs. If not, then go back to step four again and re-assess existing and alternative data collection methods.

Step 7: Assessing the usefulness to decision-making

Ask yourself again, ‘Does this KPI data help us make better decisions?

Step 7 is another sense check to make sure you can act on the data you are about to collect. You want to be sure that you are only developing KPIs that help you to make better informed decisions. If the data you collect is ambiguous then you may need to interpret it so you can act upon it.

If you are not going to make any decisions as a result of the data, or the interpretations of the data are too difficult, then the KPI is pretty worthless and you should find an alternative.

Step 8: Creating awareness of cheating

Ask yourself, ‘How easy will it be for people to hit the benchmark or target for this KPI without actually improving performance?’

You need to consider whether or not it’s going to be easy for people to manipulate the KPI. Normally KPIs have targets, and these targets can invite people to cheat or at least get very creative in how they collect and report on data.

The best way to avoid this scenario is to create an awareness of cheating and the potential for manipulation, so you pre-empt the problem and ensure your KPIs are hard to fudge. If it’s too easy to cheat or there are too many ways to cheat, consider going back to step four and finding a data collection method that is less susceptible to manipulation.

Step 9: Are the costs and effort justified?

Ask yourself, ‘Is the insight derived from this KPI worth the cost and effort required to get it?

Once you’ve identified and developed your KPIs and you know what you could gain from the KPI, but also know what’s involved in terms of data collection and interpretation, you will be in a much better position to ascertain whether the costs and effort associated with each KPI are justified. Implementing and using KPIs can be very expensive. For example, estimates suggest that the ‘Best Value’ measurement initiative introduced by the UK Government added £29 million (almost US$50 million) a year to the running costs of the police force due to the increase in data collection and reporting.

If the KPI is expensive, time-consuming or you’re having trouble justifying it, then go back to stage four and consider whether you can get the same information more efficiently.

Step 10: Collecting the data

Ask yourself, ‘OK, what are we waiting for?’

If your KPI has reached step ten intact and the KPI benefits are clear, start collecting the data and use the KPI.

Following this ten-step process every time you are considering a KPI or want to review your existing KPIs ensures that you can rest easy knowing that you have designed a suite of relevant and meaningful KPIs.

Making it work: the ten-step template in action

I was working with a major hotel chain when we applied the KPI decision template to improve existing ways to measure performance, and they had one ‘Aha!’ moment after the other.

Let’s look at the measurement of staff as an example. In the past, the company happily conducted an annual staff survey. Here are the various steps and some key learning points:

• Step 1: Linking KPIs to strategic objectives.

  The management team acknowledged that staff were absolutely vital to the success of the business, but the various questions on the staff survey were not tightly liked to any strategic objectives. The biggest staff-related strategic objective for this hotel group was: To have engaged staff that are proud to work there.

• Step 2: Identifying the unanswered questions.

  Again, the team realised that the findings from the staff survey would often throw up new questions, but rarely systematically provide answers to the key business questions. The key questions were: ‘To what extent to our staff feel engaged and proud?’ as well as ‘What do employees particularly like and dislike about working here?’

• Step 3: Isolating the decisions to take.

  What the team realised was that they had lot of data and numbers but little actionable insights, especially around the things staff particularly liked or disliked. The aim was to create metrics that could lead to corrective actions.

• Step 4: Checking for existing data and methods.

  The existing staff survey was very comprehensive but was lacking the information on the things employees particularly liked and disliked.

• Step 5: Collecting meaningful data in time.

  The company only conducted the staff survey once a year, leaving a lot of time to pass between any measurements. We changed this to quarterly surveys (of a quarter of the workforce each time). The traditional survey was replaced with a three-question survey:

  • Would you recommend this company as an employer to a friend?
  • What do you particularly like about being employed here?
  • What do you think the company could improve to become a better employer?
• Step 6: Assessing the usefulness to answering the questions.

  The new set of questions was now tightly linked to the KPQs and was providing the answers to the most critical staff-related business questions.

• Step 7: Assessing the usefulness to decision-making.

  The balance between the numeric input from the first question and the open-ended feedback from the second and third questionS provided rich insights that could easily be turned into corrective actions.

• Step 8: Creating awareness of cheating.

  The management team had few concerns about cheating when it came to the staff feedback, but felt that the new approach improved accuracy and eliminated some of the dangers around bias – for example, where the data is collected only once a year and might coincide with unusually positive or negative headlines or events.

• Step 9: Are the costs and efforts justified?

  The new approach reduced the costs significantly because the company was able to bring the data collection back in-house, instead of paying an external survey company a lot of money each year to collect and analyse the data. In addition, the time staff spend completing the survey has gone down, and so has the time it takes to analyse and interpret the data.

• Step 10: Collecting the data.

  The hotel group has now been running the new survey for a number of years and is very happy with the insights it is getting. Each quarter, a new set of initiatives is launched to improve staff engagement and overall the survey is showing increasing levels of staff satisfaction.

Deciding on How to Collect the Data

Once you have identified suitable KPIs and applied the decision-making framework to ensure they are the right KPIs for your needs, you need to decide how you will collect the data.

Deciding how to extract actionable insights and knowledge depends on your answers to three interdependent questions:

1. What data do you need to meet your information needs?
2. Does the data exist in the right format?
3. If you have not got the data in the right format then what is the best way to obtain that data?

You need to ensure that you collect the right data and the right quality of data so that you can answer you KPQs accurately. An effective data-driven strategy is built on your ability to collect, analyse, and turn data into meaningful information and insights. Poor quality or inappropriate data will compromise your ability to make important decisions, implement the right strategies for your business and improve performance.

Identifying types of data

When it comes to data collection the two main types are usually called quantitative (numeric) and qualitative (non-numeric).

As a general rule qualitative data is particularly useful for establishing theories and gaining a deeper understanding of specific issues, while quantitative data is particularly useful for testing theories and assumptions.

Applying quantitative methods

The most common assumption about KPIs is that they are numerical. If you were to ask 100 executives for an example of a KPI, my guess is that all of them would immediately name a quantitative KPI such as Return On Investment. Quantitative data is by far the most commonly collected in business, because it is easier to collect. Plus it’s also easier to translate and interpret. It’s much more black-and-white.

If your KPQ requires that you classify features, count them, and then construct a statistical model to explain what you observed, then you probably need to collect quantitative data. Quantitative data is usually collected automatically via normal operations such as customer service conversations, or through a well-designed survey.

Creating good surveys

Questionnaires and surveys are popular methods for collecting valuable performance data. Although they can be extremely expensive, they don’t need to be. Once you have designed the survey, technology can distribute it electronically to thousands of employees or customers around the world, at minimal cost.

There are, however, some rules to follow when designing your survey to ensure that you collect meaningful data that you can then use to improve results:

• Explain the purpose and potential benefit of the survey. If recipients understand why they are being asked to complete the survey and what difference the results might make to their lives then they are more inclined to take it seriously, engage with the questions and provide useful information.
• Make the purpose of each question clear so it’s obvious why you’re asking it. Most people are responsive to questions when they appreciate why it’s being asked and can appreciate the relevance and importance of the question.
• Use clear, simple language. That means no overly complex ‘business-speak’ or jargon.
• Ask one question at a time. Don’t roll questions together – this can be irritating if the recipient wants to answer them differently. For example I saw an HR survey that asked recipients to score from 1–5 ‘How successful are we are linking competency assessment to business strategy and the evaluation of individual performance.’ They are two different questions.
• Make it as short and easy to complete as possible. Only ask questions that you genuinely need the answers to.
• Have your survey professionally designed so that it looks attractive and includes plenty of white space. If the survey looks unprofessional or cluttered then it will actively deter people from completing it.

The weaknesses of quantitative data

The biggest weakness of quantitative data is that it can often lack detail and context.

For example a section of your staff may indicate they are disengaged but the survey won’t reveal why. And without the why, you don’t have the insight necessary to change the situation.

Understanding qualitative methods

You often need to collect qualitative data after completing the initial survey to dig deeper into the quantitative responses. If, for example, your staff survey identified disengaged employees, a further qualitative approach would help to shed light on why they were disengaged. By analysing this additional, richer data in context you will understand how these employees feel and what is driving their behaviour. And it is those insights that can help identify a solution.

In the following sections, I discuss the most popular methods for collecting qualitative date.

Conducing qualitative surveys

Instead of asking recipients for quantitative data – such as rating their level of agreement with a specific statement, for example – a qualitative survey asks open questions to collect more detailed, albeit less structured, information. For example, you might ask your customers to describe their last purchasing experience with your company.

To save time, money and potential survey overkill, many companies are now incorporating both quantitative and qualitative elements into their surveys. Allow recipients to provide additional feedback should they want to, so they have the opportunity to raise issues that are important to them but do not appear on the survey.

Focusing on focus groups

Focus groups are facilitated group discussions of between five and 20 participants who are asked to share their ideas, experiences and opinions.

Focus groups offer an interactive element which can result in better data. When people get together and share their experiences, this debate can often trigger additional insights and information that may be missed in a survey. As a result this approach is particularly useful for assessing highly subjective indicators such as customer experience, customer or staff engagement, team-working, organisational culture, or trust. They can, however, be quite expensive and time-consuming so you’d better make sure you really need the data, and that it will drive decisions and improve performance.

The mystery of mystery shopping

First used in the mid-1980s by the more pioneering retail organisations to assess customer experience, mystery shoppers are now used extensively in the service sector, for example in the hospitality, banking and financial services industries.

As the name would suggest, a company employs or outsources ‘secret shoppers’, posing as genuine clients or customers, who then buy a product or service and report back to HQ about the experience. This approach is useful because you don’t have to impose on your real customers to get an idea of the customer experience. Plus you can control what you are trying to assess more easily. For example, you could ask your mystery shoppers to focus on a particular area of the customer experience, such as complaint handling. You can also use this approach for other internal performance assessments, such as organisational culture or atmosphere.

Conducing peer-to-peer assessments

Gathering information via peer-to-peer assessment can be very insightful because you are effectively getting people who work closely together to assess each other – either openly or anonymously. Done well, it allows people to learn from each other and to consider their own performance from the perspective of their colleagues. However, peer-to-peer assessment is a recipe for disaster if done badly. It’s imperative that the process is put in the right context so that the people involved do not get defensive.

Peer-to-peer assessment is particularly useful for gauging things like:

• Trust
• Knowledge and experience
• Teamwork
• Relationships

Observing situations

Observing situations or activities without interfering in or manipulating the environment can also provide a great deal of qualitative data. The benefit of this approach is that you don’t disturb or interrupt anyone. It can also provide additional, more subtle, insights because the person observing can take in and make sense of the entire experience through all the senses. As a result, observation can provide a more holistic understanding of what’s being observed than other data collection methods.

Observation outputs can be score sheets, checklists, reports or video or audio recording. This approach can be particularly useful when assessing organisational culture, employee skill and experience, emotional intelligence, and creativity.

Putting bullets on a board

Many of the methods above have been around for a long time. Changes in technology, however have also created addition methods for data collection.

For example, online bulletin boards allow a discussion (or thread) to develop over time and can be a rich source of qualitative data. By regularly interacting with participants on the bulletin board you can assess how an idea, concept or sentiment grows and changes over many weeks or even months. This is also true for blogs and social media. In a world where just about everything is shared there is, by definition, a wealth of qualitative data online if you know where to look.

The weaknesses of qualitative data

A biggest challenge with qualitative data is that it’s not neat, formulaic or structured. As a result it can be much harder to collect, analyse and interpret.

It is also more susceptible to bias than quantitative data, because it is much easier for a person to put a particular spin on the collection of perceptions or observations than it is on hard numbers. Numbers are more definitive and objective: They are what they are. Qualitative data is subjective and open to interpretation.

To minimise bias make sure you challenge the data and the person collating the data to understand the assumptions that sit behind their findings.

Combining data to improve insights

Both qualitative and quantitative methods are important. Using one without the other can obscure facts and introduce bias and assumption rendering the whole data collection process useless.

You need to accept that KPIs will not capture the whole truth about your business. They are indicators, not necessarily facts. After all, if they were facts then we would be talking about KPFs and not KPIs! KPIs indicate a certain level of performance, but they don’t measure that performance. For example, a customer service indicator will give you an indication of how your customers feel about you but it won’t measure total customer satisfaction or provide a complete picture. If you were to take the Mensa I.Q. test, it would give you an indication of a particular type of intelligence but it doesn’t completely measure all dimensions of that intelligence.

The more data points you get – quantitative and qualitative – the more complete your picture will be. A data point is just an item of factual information gained from measurement or research. For example there are many data points around a financial transaction including how much was transferred, when it was transferred, who sent the money, who received it and what currency it was received in. Each item or piece of factual information is known as a data point.

When you also vary the data collection methods you use, you can triangulate the information. This simply means that you collect data from different data sources, use different methodologies or use different people to collect the data so you can contrast and compare what you find, helping to validate the reliability and accuracy of the information.

It is incredibly easy to unwittingly introduce bias into data collection. Ultimately the process is managed by people and people come with a whole bunch of expectations, judgments, beliefs, values and assumptions that can colour the data.

Abraham Wald was a WWII statistician who helped the air force to assess aircraft vulnerability. The strategy was – find out the areas on the aircraft that were most vulnerable to enemy fire and reinforce those areas. Makes sense! Every airplane was then examined for bullet holes to establish what parts of the plane were hit the most. Based on the data the air force then concluded those areas most hit should be reinforced. Again, makes sense … until Wald pointed out that the data was flawed. The only planes that could be examined were the planes that made it back to base. The fact that the aircraft made it home proved that the bullet holes, regardless of how many, were not fatal in those locations. It therefore probably made more sense to reinforce all the areas where there were no bullet holes because those planes never made it home!

The big data challenge

In the last two decades two things have changed the game when it comes to data collection. The first is that computer storage and processing power has increased exponentially. If you think back to your first desktop computer – go on, it’s not that long ago – the smartphone in your pocket is now more powerful. You can buy a portable hard drive with a terabyte of storage for under $100. You could write books 24 hours a day seven days a week for the rest of your life and never come close to filling a terabyte of storage! Plus technology has also made it possible to process and analyse all that data for the first time.

The second game changer is the amount, type and quality of data being generated. Consider the explosion of social media and our love of sharing (and over-sharing) information. Data is everywhere, in words, numbers, images, video, audio, status updates, tweets and so on. The amount of data we now produce is genuinely staggering, hence the term big data.

Potentially, this messy, unstructured data offers you many more data points from which to draw your evidence. The challenge, however, is that the sheer volume of the data now available makes the task of finding the right bits even more daunting. It is now more important than ever that you develop the right KPIs, so that you can pluck the useful information from the ever-expanding sea of irrelevant data.

Finalising Your KPIs: Applying the KPI Design Template

To help you design your KPIs I’ve developed a KPI design template. Ideally you should use this in conjunction with the KPI decision framework described earlier. The template helps you to eradicate the ambiguity, ambivalence, and inconsistency that can so often creep into data collection and KPI reporting.

If your KPIs are to become the basis for strategic execution, growth, learning and evidence based decision-making everyone must understand what the KPIs mean, why they are needed, how reliable they are, where the data comes from, how it’s collected while also identifying the KPI targets.

Use this design template to develop completely new KPIs that deliver the answers you need or improve the effectiveness of your existing

The basics

The first four elements of the KPI design template address the basics of each KPI and help to put it in context.

Strategic Objective

It’s always best to clearly specify to which strategic objective the KPI relates, so that everyone looking at the KPI immediately appreciates its relevance.

If appropriate you can also identify the person(s) or function(s) responsible for the management and delivery of the strategic objective that the KPI is assessing. This may be an individual executive or employee, or a team of people. Clarifying ownership in this way allows you to know who to call in the future should you need to discuss performance, or fine-tune the KPI.

Audience and Access Rights

Here you define the primary audience for this KPI – basically, who will see the data and who will have access to it.

Sometimes it is possible to define a primary and a number of secondary audiences. For example, the primary audience for financial information might be the senior leadership team, and secondary audiences might include shareholders, analysts and other functional managers within the business.

Key Performance Question (KPQ)

For each KPI, state the KPQ that the indicator is helping you to answer.

Again this helps to provide context around why this particular indicator is being introduced and on which specific issue it is going to shed more light. It puts the KPI in context and helps keep people engaged in its on-going measurement.

How will and won’t the data be used?

Specify how the KPI will be used; for example, share the decision(s) the KPI is helping you make. This provides even greater context, so that everyone who uses the KPI or comes across the KPI is clear about how you plan to use the information and evidence it provides.

This is especially important if you are introducing a suite of new KPIs, because it helps to reassure everyone involved that every one of the KPIs has a very specific purpose and is not just added to make the initiator look good!

Another part of this section is to define how the KPI will not be used. Sometimes, people are scared to report on measures because they fear negative results could be used against them. Here, you can say that the KPI won’t be used to determine the performance of individuals and won’t be linked to bonus payments.

Completing your KPI Template

The rest of the KPI template covers the more technical aspects of the data collection. It’s essential that you consider the strengths and weaknesses of the different data collection methods and how appropriate they are for you and your business.

As designer of the KPI you should include the elements in the following sections.

Indicator name

Every KPI needs a name so that you can discuss it collectively and everyone knows exactly what is being discussed. Choose a name that clearly explains what the indicator is about.

If the KPI you have chosen to measure already has a name, make a special effort to ensure that everyone is on the same page. Too often in business we use language or jargon that means something different to different people. This can cause problems, so double check that everyone is talking about the same thing.

Data collection method

Identify and describe the data collection method you are going to use for each KPI. It’s important to keep the strategic objective and KPQ in mind when you do this. Too often the decision about which data collection method to use is an automatic response based on traditional methods, past experience or plucking the latest one discussed in management journals. It is far better to really engage with the challenge and consider the strengths, weaknesses and appropriateness of different data collection methods.

Data collection methods include surveys, questionnaires, interviews, focus groups and collection of archival data.

Targets and Performance Thresholds

Define a target or benchmark for each indicator – for example, grow our revenues by 20 per cent over the next 12 months. Here you can also outline the performance thresholds, that is, when performance levels are judged to be good or bad. A common way to do this is to institute a traffic light system where red represents bad performance or target missed, amber (or yellow) is flagging up minor issues and green is good performance or on target. Some companies prefer to add another one – usually blue – to indicate target exceeded, which in most cases is a good thing but could indicate too much effort is put into one area maybe to the expense of another.

---

Scale Types

Many different types of scale exist:

• Nominal: Differentiates between items based on names or categories such as gender, nationality or language.
• Ordinal: Differentiates between items based on rank order such as determination of more or less, star ratings etc.
• Interval: Differentiates between items based on intervals, where the steps of the interval are the same, but you can’t claim that 10 is half of 20 such as temperature in Fahrenheit or Celsius.
• Ratio: Is the highest form of measurement scale, where we look at an interval scale that has a true zero, so that you can claim that 10 is half of 20. Example include length, time, or temperature in Kelvin.
• Likert scale: Determines the extent to which the respondent agrees or disagrees with a statement. A typical Likert scale is:

• 1 = Strongly disagree
• 2 = Disagree
• 3 = Neither agree nor disagree
• 4 = Agree
• 5 = Strongly agree

The scale you choose is very important because it has implications on how you can use the data. For example, a nominal scale just tells you if something is one thing or the other – it doesn’t reveal any order or relative size. An ordinal scale adds more depth by telling you whether something is bigger or better than another although it won’t tell you how much bigger or better. The Likert scale can be used to determine how much bigger or better.

Different options exist to extend the classic 5-point Likert scale and to change between odd-numbered and even-numbered scales. The danger with odd-numbered scales is that there is, by definition, a middle value which represents a neutral, undecided or neither agree or disagree response. This can become an easy option or cop-out for respondents. Even-numbered scales avoid this scenario because they don’t provide a neutral response. As a result, even-numbered scales are often called forced-choice response scales because they force the respondent to get off the fence one way or the other. Forced-choice questions are useful tools if respondents are reluctant to state their preferences.

---

Formula, scales and assessment criteria

It’s important to specify the formula, scale or assessment criteria that will be used for the KPI so that you create uniformity around the data, and so that everyone is talking the same language. Is it possible to create a formula? Is it an aggregated KPI (where for example overall sales revenue is aggregated by adding all the sales revenues from the business units) or index that is composed of other indicators (for example, a quality index could be composed of waste levels and rework levels)? Are you using a scale?

Source of the data

Your KPI template should specify where the data is coming from so that people using the KPI can be assured of its reliability and validity.

For example, if you decide that the best way to collect customer sentiment is through interviews, questions may arise about how reliable that data is. People are not always as honest as they could be in that environment, so interviews in this context are likely to give a distorted picture of what’s really going on.

Data Collection Frequency

You need to specify how often the data for the KPI will be collected and coordinate the dates when data is collected.

Some KPIs require data to be collected continuously. Others specify hourly, daily, monthly, quarterly or annual collection. It’s important to know why you are choosing the frequency so you can make sure the data is only collected as often as it’s actually needed. If you wanted to lose weight you would make the best decisions if you knew what you weighed every day; you wouldn’t gain anything if you chose to weigh yourself every hour. It would just waste time and energy.

The reporting requirements for each KPI will also influence frequency of collection. If, for example, the data has to be reported at the end of each month, then it makes sense to schedule collection so that there is enough time to collect the data, chase people where necessary, analyse it, aggregate it, solve any issues and deliver the report while still ensuring the data it contains is as recent as possible.

It also makes sense to coordinate collection dates. Too much data collection is ad hoc and uncoordinated. The result is it either doesn’t get done, is the first thing to get bumped off the to-do list when someone gets busy, or it wastes too much time. Wherever possible, coordinate and schedule the data collection so as much of it as possible gets done at the same specified time. This is much more efficient than scattering collection over a longer period, and if all departments are collecting data at the same time it is much easier to get a valid snapshot of performance across different areas of the business. This is much harder if data is collected at different times from different departments.

One of the biggest mistakes companies make in performance assessments is in not collecting data frequently enough. For example, most large companies conduct staff surveys once a year, which provides a single snapshot of staff sentiment. A far more insightful way to collect that data would be to survey 10 per cent of the workforce every month. Everyone is still only surveyed once a year but this approach allows you to plot trends because there are 10 data points (10 per cent for 10 months) instead of 1 data point.

Data Reporting Frequency

You need to specify when and how often the data for the KPI will be reported. It might go into the weekly or monthly performance report or be updated on the performance dashboard on a bi-weekly basis.

It makes sense to coordinate the data collection and reporting frequency to ensure the data you are reporting is as current and up-to-date as possible. For example, you don’t want to end up in a situation where data is collected in January and reported at the end of the year.

Who measures and reviews the data

Your KPI framework should also specify the individual or job title of the person responsible for the data collection and data updates. It is always preferable to name a specific individual because the job is more likely to get done that way.

The owner of the KPI can be a named employee or business function, or, increasingly, an external agency; many businesses are outsourcing data collection for some KPIs. This is especially common for KPIs connected to qualitative issues such as customer satisfaction, reputation, brand awareness and employee engagement.

Make sure you also clarify whether there are any review or sign-off cycles. Often one person will be responsible for collecting the data, including data input, and another person will be responsible for cross-checking or signing-off the data before it is released.

Expiry and revision dates

Your KPI template should always include an expiry date or revision date. KPIs are sometimes only needed for a specific period of time, perhaps during a particular project or restructuring process. Without an expiry or review date these KPIs can continue indefinitely, causing unnecessary work.

Even if indicators are not time- or project-specific they should be assigned a review date which is in the diary of everyone involved. When that date rolls around you re-visit the KPI to ensure it remains relevant and useful. If it’s no longer relevant then delete the KPI.

How good is the indicator?

Following the KPI decision framework allows you to develop the right KPIs and only the right KPIs. The final part of the process establishes how good your KPI is in terms of cost, completeness and ability to manipulate.

How much will it cost?

Introducing and maintaining a KPI can be expensive. It follows, therefore, that you need to estimate the costs involved so you can make sure it’s all worth it. That estimate should also be included in your template to remind you of that assumption should anything change that would require you to re-evaluate the validity of the KPI.

Costs can include the administrative costs or outsourcing costs for collecting the data, as well as the efforts needed to analyse and report on the performance.

How complete is this indicator?

By now you will have worked through all the key aspects of the KPI and will be able to determine how confident you are that the KPI will help you answer your KPQs and support your decision making.

For financial KPIs, your confidence is probably high, since long-established and widely used KPIs are used to measure performance in this area. However you may not be as confident measuring intangibles such as organisational culture, employee competencies or brand image and reputation. It’s important to acknowledge this and assess your confidence level, because doing so forces you to think about how confident you are that the KPI will actually measure what it was designed to measure.

You can express your confidence in the KPI in a variety of different ways:

• As a percentage from 0 to 100 per cent
• As a grade from 1 to 5
• As a colour code such as red, amber and green
• As symbols such as smiley faces or frowns.

In whatever way you choose to express your confidence level, include a brief written comment to clarify the level of confidence and explain the limitations of the KPI.

Being aware of any unintended consequences

In the development process you will already have thought about the various ways the KPI can potentially be manipulated or how people can cheat. Make a note of these. If you discover new ways, add them to the list. Reflecting on possible KPI dysfunctions allows you and the people using the KPI to consider even better ways of collecting and assessing performance. Calling these behaviours out at the start can help to stop people from trying them and alert everyone to monitor for those behaviours.

Together, the KPI decision template and the KPI design template provide powerful tools to make any KPI development project a success. In the sample KPI template you can find two practical examples of how the template could be completed for the NPS (Net Promoter Score) and profit margin KPI. The KPI design template is shown in tabular form in Table 5-1.

Table 5-1 Sample KPI Design Template

	Example 1	Example 2
Strategic Goal: Name the strategic objective (from the strategy map), which is being assessed with this indicator.	Grow Customer Satisfaction (Customer Perspective)	Grow Our Profits (Finance Perspective)
Audience / Access Name the key audience for this indicator and clarify who will have access rights to it	Board of Directors and Marketing Team	Board of Directors and Finance Team
Key Performance Question(s): Name the performance question(s) this indicator is helping to answer.	To what extent are our customers satisfied with our service?	To what extent are we generating bottom-line results?
How will and won’t this indicator be used? Describe how the insights this indicator generates will be used and outline how this indicator will not be used.	The indicator will be used to assess and report on our customer success internally. It will not be used to assess performance of individuals or to determine bonus payments.	The indicator will be used to assess and report financial performance internally and externally. It will also be a key indicator to determine executive pay.
Indicator Name: Pick a short and clear indicator name.	Net Promoter Score	Net Profit
Data Collection Method: Describe how the data will be collected	The data will be collected using a mail-based survey.	The data for the net profit metric is collected from the income statement (or the finance and accounting system).
Assessment / Formula / Scale Describe how performance levels will be determined. This can be qualitative, in which case the assessment criteria need to be identified, or it can be numerical or using a scale, in which case the formula or scales with categories need to be identified.	Using a 0-10 scale (Not at all likely to extremely likely) participants answer: How likely are you to recommend us to a friend? NPS = pecentage of Promoters (score 9–10) – 5 of Detractors (score 0–6)	Net Profit ($) = Sales revenue ($) – Total Sales ($)
Targets and Performance Thresholds Identification of targets, benchmarks, and thresholds for traffic lighting.	55 per cent by the end of 2020	$1,250,000 by the end of 2020
Source of Data Describe where the data will come from.	Survey of existing customers	Finance and accounting system
Data Collection Frequency Describe how frequently is this indicator will be collected. If possible, include a forward schedule.	Monthly data collection – sampled 10 per cent of our customer data base	Weekly
Reporting Frequency Outline how frequently this indicator will be reported to the different audiences (if applicable).	Monthly	Weekly
Data Entry Name the person or role responsible for collecting and updating the data?	Ian Miller – Marketing Assistant	Joe Blox (Finance Clerk)
Expiry / Revision Date Identify the date until when this indicator will be valid to or when it will have to be revised.	24 months	Target to be revised annually
Validate your KPI
How much will it cost? Estimate the costs incurred by introducing and maintaining this indicator.	Costs are significant, but cheaper than a traditional customer satisfaction survey.	The costs of producing the net profit measure are low because the data is readily available.
How complete is this indicator? Briefly assess how well this indicator is helping to answer the associated key performance question and identify possible limitations.	It provides us with a nice simple number, but the data should be supplemented with unstructured feedback about: What is particularly good? What could be improved?	Net Profit is one of a range of profitability metrics. However, on its own it will not give us the full picture and can lead to short term thinking. It will need to be seen over time and in the context of other measures such as revenue, profit margin, operating profit, return on assets and return on equity.
Possible unintended consequences Briefly describe how this indicator could influence the wrong behaviors or how people could cheat on this KPI.	People could possibly influence customers before they take the survey or they could select customers who are likely to respond positively.	The danger with net profit is that people could cut costs to the detriment of long-term performance but deliver positive shot term results.