Chapter 12
Determining Measures That Will Work in Your Organization (Stage 5)

In this important stage (Stage 5 in the methodology) we need to first understand how to determine measures, how to record them, understand the importance of sorting the wheat from the chaff, and agree that measures are best found at the workface by teams rather than the traditional top-down approach of the balanced scorecard. Having made this agreement, we need to understand how to sell the KPI methodology to the employees as well as explain how the workshops are run.

How to Derive Measures: An Overview

Many performance measures are created from a flawed process. Numerous methodologies, including the balanced scorecard, appear to simply say the measures are a by-product of the exercise. Frequently the task of finding measures is a task carried out at the last minute by staff members who do not have a clue about what is involved in finding a measure that will receive the appropriate behavioral response. As mentioned in Chapter 2, it is one of the myths of performance measurement that appropriate measures are very obvious. There are a number of common reasons why organizations get their measures radically wrong as set out in Exhibit 12.1.

Exhibit 12.1 Common Reasons Why Organizations Get Their Measures Radically Wrong

“SMART” is not the answer To be under the misconception that as long as a measure is SMART—specific, measureable, attainable, realistic, and time sensitive—it will do. This, of course, ignores the fact that the measure may not be linked to the critical success factors of the business and that its dark side may be very damaging.
Cascading does not work The cascading down of performance measures, where one measure is broken down into its component parts as it goes down different teams (e.g., you start with return on capital employed and then say what measures made this up, and so on down).
Training is essential Giving teams the task of finding measures without any training and placing them in the organization's balanced scorecard application.
Top down does not work Giving the task to a few accountants or performance-management specialists who complete this task in the spare moments they have.
Tying KPIs to pay will fail Tying KPIs to performance-related pay. If you do this, KPI stands for key political indicator, rather than key performance indicator. As mentioned earlier, good performance with KPIs should be seen as a “ticket to the game,” a given, the reason why you are employed; thus, there is no need to incentivize them.

An overview of the important work that needs to be done in order to arrive at measures that will work in your organization is set out in Exhibit 12.2.

Exhibit 12.2 Summary of What Needs to Be Done in Order to Arrive at Measures That Will Work

Task Description
Locate success factors Determine what your success factors are. This has been covered in Chapter 11 (Stage 4).
Ascertain CSFs Map the relationships to find which success factors have the most significant impact—the critical success factors. This has been covered in Chapter 11 (Stage 4).
Train a small KPI team Train a small team in how to derive measures led by the chief measurement officer. This has been covered in Chapter 9 (Stage 2).
Selling the KPI system to all employees This important step is covered in Chapter 10 Stage 3.
Teams design appropriate measures Teams review the critical success factors and design appropriate measures for their team.
Teams record performance measures in a database Record all measures collected from the workshop sessions in a database, indicating the key features, such as description of measure, suggested measure name, the critical success factor the measure influences, and measurement frequency.
Sort the measures Sort the wheat from the chaff by having an experienced KPI team member review every measure.
Determine measure name and how it is measured Finalize the appropriate names for all the measures that will be used and clearly think through how the measures will be utilized.
Find the KRIs Ascertain the KRIs that we will need to report to the Board.
Ascertain the team performance measures Teams ascertain measures from relevant CSFs.
Find the KPIs Ascertain the winning KPIs and commence testing of them.
Measures Gallery Hold a “Measures Gallery” as suggested by Stacey Barr,a which is an open session where staff are invited to share their views on the measures which have been displayed on the walls of the room.

aStacey Barr's Practical Performance Measurement Using the PuMP Blueprint for Fast, Easy and Engaging Performance Measures, 2014.

Ascertain the Team Performance Measures

This is a vital step in performance improvement and was step 8 in the first and second editions. The appropriate team performance measures will help teams to align their behavior in a coherent way to the benefit of the entire organization. This is achieved because teams are focusing on those performance measures that are linked to the organization's CSFs. Team performance measures will be comprised mainly of relevant RIs, PIs, and some of the organization's KPIs (e.g., a late plane's measure would have been monitored by the front desk, engineering, catering, cleaning, etc., but not the accounting team).

Although management often tends to become focused on achieving KPI introduction at the global, organization-wide level, in reality the critical issue is getting these KPIs embedded in the teams that need to take corrective action 24/7. Thus, it is at the team level—level 4 in Exhibit 12.3—that significant and sustainable performance improvement can be achieved through the use of performance measures.

c12ex003

Exhibit 12.3 Interrelated Levels of Performance Measures in an Organization

Why Team Performance Measures Are Critical

Every CEO wishes that employees' day-to-day work would align itself with the organization's strategic objectives. Yet this is seldom the case. Why does your marketing team measure all customer satisfaction infrequently when the CSF in that area might be “increased repeat business from key customers”? Surely you should be measuring the satisfaction of key customers regularly and ignoring those customers you could do without. Why does dispatch do the same quality control and timely dispatch procedures for all customers, when it is your key customers that should get extra checks at the expense of those customers that you would be better off losing? The answer lies with the fact that we have not communicated the critical success factors to staff, nor have we worked with them to select the measures that stem from these CSFs. Once we have performed this, a magical alignment can occur between effort and effectiveness.

Key Tasks for the Roll Out of the Team KPI Workshops

The KPI team will need to incorporate these tasks within the work it performs in stage 5.

Task 1. Holding the Team KPI Workshops

The KPI project team must provide training and assistance to all teams so that they are equipped to select their own performance measures that are consistent with the organization's CSFs. It is a good idea to bring a number of teams together at the same time, as they will learn from each other's different views. Some workshop groups have up to 80 attendees doing the team performance measures exercise. Each team is broken up into a discussion group of four to seven staff members. To assist I have provided the proposed agenda for the KPI workshop and the breakout work group sessions in PDF format for you to download.

Encourage balance in the teams' performance measures. If the CSFs are clearly defined and related to the six BSC perspectives (customer focus, financial performance, learning and growth, internal process, employee satisfaction, and environment/community), then team performance measures developed in this context generally will reflect the required balance.

In the opening presentation, address the issues and perceptions raised in the employee survey that are important to your audience through small-group briefings, spelling out the workplace vision. The briefings are used to explain the purpose and use of KPIs, address any concerns and ground rules, and discuss the way forward. Employees often are concerned that performance information will be:

  • Collected on individuals and held against them (e.g., for disciplinary purposes)
  • Filtered both in content and distribution (e.g., “They only show us information when it suits their purposes”)
  • Used to allocate blame for performance problems

Task 2. Promote an Appropriate Mix of Past, Current, and Future-Looking Measures

Current measures are measured 24/7 or daily, and future measures are the record of a future date when an action is to take place (e.g., date of next meeting with key customer, date of next product launch, date of next social interaction with key customers). (See Exhibit 12.4.) Key result indicators and result indicators will all be past measures, whereas performance indicators and key performance indicators are now categorized as either past, current, or future measures. You will find that most of the KPIs in your organization will fit into the current or future categories. The previous debate about lag (outcome) and lead (performance driver) indicators is now dispensed with. As pointed out in Chapter 1, lag and lead indicators do not work and serve to confuse.

Exhibit 12.4 Taking a Past Measure and Restating as a Current and Future Measure

Past measures
(last week / fortnight / month / quarter)		 Current measures
(24/7 and daily)		 Future measures
(next day / week / month / quarter)
number of late planes last week/ last month planes over 2 hours late (updated continuously) number of initiatives to be commenced in the next month, months two and three to target areas which are causing late planes
Date of last visit by key customer Cancellation of order by key customer (today) Date of next visit to key customer
Sales last month in new products Quality defects found today in new products Number of improvements to new products to be implemented in next month, months two and three

Task 3. Permit Team Performance Measures to Evolve

Virtually no team will achieve a perfect set of performance measures at its first or even its second attempt. Further, once a set of performance measures exists, individual indicators may need to vary as the team improves performance and then moves on to focus on other problem areas.

Task 4. Assess the Level of Understanding That Attendees Have of the Organization's CSFs

During the workshop we need to assess the level of understanding of attendees regarding the organization's CSFs. This should occur toward the end of the day. To assist I have provided a worksheet in PDF format for you to download.

Task 5. Never Lose Sight of Team Ownership

Remember that the primary purpose of team performance measures is to assist and help the team to improve their performance. It follows that their performance measures represent what they want to collect in order to contribute to improvement in the identified CSFs. The KPI team needs to steer them gently if they are off course.

Task 6. A Maximum of 25 Performance Measures for a Team

As a guide, 25 is probably the upper limit of performance measures that a team should select for regular use. Any more than this number may lead to resource problems and a lack of focus. These performance measures will include some of the organization's KPIs. Some teams may have up to three organizational KPIs in their team scorecard. Many head-office teams will not have KPIs in their scorecard, as they are not relevant to them (e.g., the British Airway's accounting team would not have a late planes measure). Remember that the KPIs affect the entire organization. Thus, there are no KPIs specific to one team.

To assist you in delivering a successful workshop I have provided a checklist in PDF format for you to download. The KPI team, with the facilitator, should amend this checklist before use to suit the organization and desired approach.

Designing Appropriate Measures

When designing measures there needs to be a very structured process. To assist with the designing of measures, here are some useful rules:

  • Make sure you have conducted a training session delivered by the now experienced KPI team on how to word the performance measures and how to make them measurable.
  • Use a mix of oracles and young guns in the designing measures session.
  • Everybody is considered an equal during the session; thus, ensure that the most senior member does not act as the facilitator as this sets up the ritual of a meeting.
  • You need to design measures from the critical success factors, which are fine-tuned statements and easy to understand by all those involved so they will share the common goal. When you design a measure from a well-worded critical success factor, you are already focused; in other words, you are looking through the sights of a rifle.
  • Many measures in the workshops will be of little use. The sorting out of wheat from the chaff is carried out by the KPI team, and the process is covered in a later section of this chapter.

You start off with a chosen CSF and then you ascertain the results you would expect to see if the critical success factor is working correctly. Often the results come from the words used in the critical success factor. I tend to aim for four to five results. For example, if we were to ascertain measures for the critical success factor “maintaining a safe, happy and healthy workplace” (see Exhibit 12.5), the results could be: staff happy with their work, workplace is safe at all times, healthy working conditions at all times, there is a positive “go for it” culture.

Exhibit 12.5 Process to ascertain Measures from a Critical Success Factor

I ask what aspects or areas should be measured in this critical success factor. The aspects are only trigger points and often include duplication. All suggestions for aspects should be treated as good ideas. There is no editing or disputing an idea. Often, the most unusual idea can give birth to a major discovery.
image
I always aim for at least five aspects. Look for key words in the CSF wording.
image
We start ascertaining the likely measures. Which include:
  • # of recognitions given last week by department
  • # of CEO recognitions planned for next week/next fortnight
  • # of incidents or near misses (immediate reporting)
image
The next stage is to mark the measures that are:
  • Past measures (# of recognitions given last week by department)
  • Current measures (# of incidents or near misses)
  • Future measures (# of planned celebrations in next week / next fortnight)
image

With each aspect, you then commence thinking of likely measures. Ask the attendees in the session, “What would good performance look like?” Using their answers, ask them, “What measures would show this good performance best?” Then ask, “What would poor performance look like?” In response to their answers, ask them, “What measures would give advance warnings of this negative performance?”

Before you move on, you need to ensure that you have a mix of past (P), current (C), and future (F) measures (See Exhibit 12.5).

Recording Performance Measures in a Database

The database of performance measures is fed from a number of sources. The KPI team will have gathered and recorded performance measures from:

Now it is time to use these measures to help the staff ascertain their team measures and, in addition, to trap all the new performance measures generated from each team workshop.

Performance measures identified need to be collated in a database. This database needs to be up-to-date, complete, and made available to all employees to help support their understanding of performance measures and to assist with their selection of their team measures. The suggested fields are set out in Exhibit 12.6.

c12ex006

Exhibit 12.6 Performance Measure Database Layout Example

During all workshops ensure the database is updated before they go to any break. I normally use a simple word table as the input form, which is updated on an employee's laptop and passed over to the KPI team via a USB stick. At this point we only need to record name and time zone. The rest can be added once the measure has passed through some tests.

Key Tasks for Recording Measures in a Database

The KPI team will need to incorporate these tasks within the work it performs in stage 5:

Task 1. Select a Database That Has Wide Access within the Organization and Is User-Friendly

Most organizations operate database applications, which are underutilized. The KPI team must learn to use the in-house database application and design and build a performance measure database that is easy to use. An access performance measure database is available from www.davidparmenter.com for a small fee. Open the templates icon and search for the “Database of Measures and Associated Success Factors.”

Task 2. Build the Database

The database should be built for ease of use and include sections where teams:

  • Select the critical success factors (CSFs)/success factors (SFs) that are relevant to them.
  • Can interrogate the database using keywords to see if their measure is already included.
  • Can add new performance measures (only the KPI team should have power to delete measures).
  • Record their selection of all the measures they are proposing to use.

Task 3. Populate the Database

On a daily basis, the measures that have been identified need to be input into the database to ensure that they are not lost in a mountain of paperwork. In order to maintain consistency of input, one person or a small team should be responsible for this action. An easy-to-use input form should be set up in the database to facilitate entering the measures in a timely manner.

Task 4. Train All Teams to Use the Database and to Refine the Performance Measures Constantly

The KPI team needs to train all the other teams on not only how to use the database but also the significance of each database field. This is best achieved through the rolling workshops they will be giving teams. Teams will be trained to review the database to see if any new measure has emerged that is very relevant for their team. This will be performed as part of a later step.

The team will need to refine the performance measures constantly by “peeling more layers off the onion.” In time, a clearer hierarchy of measures will develop, some will be discarded, and new measures will start emerging that will have a profound impact on the organization's future.

Task 5. Ensure That All Database Fields Are Complete for Every Performance Measure

The project team needs to review the database constantly, cleanse it of duplication, and encourage teams to look at measures that have been selected by their peers. The checklist in the next section can be used as an aid to the KPI project team, ensuring that important tasks are not overlooked. The KPI team, with the facilitator, should amend this checklist before use to suit the organization and desired approach.

To assist I have provided suggestions on how to build and maintain a database in PDF format for you to download.

Sorting the Wheat from the Chaff

Mankind has for thousands of years been able to sort the wheat from the chaff. It is thus in our DNA. We now need to use this ancient practice with all measures we have located. Leave the sorting of measures until after the KPI staff workshop and let this exercise be done by the KPI team—the in-house experts on performance measures.

I recommend the following sorting techniques:

Stacey Barr, in her book,2 has devoted a large chapter called “Designing Meaningful Measures” and I would recommend that the KPI team read and understand her views.

Determine Measure Name and How It Is Measured

Creating an appropriate name for the measure and ascertaining how they are to be measured is an important task. These two features should also be recorded in the performance measure database. This task is best done later on by the KPI team as many measures will by then have been abandoned, thus making the task easier. Barr's work in this area will be of use to a KPI project team.3

Use Pareto's 80/20 Rule When Assessing How to Calculate a Measure

Encourage teams to be practical when assessing how to calculate their chosen performance measures. It is essential that the cost of gathering the measure is not greater than the benefit derived from the measure. For many measures, staff should be encouraged to either use sample techniques (e.g., measure late invoices one week every month) or assessment techniques (e.g., estimate the number of coaching hours received last month). Pareto's 80/20 rule encourages us to measure in detail only the KPIs—for example, the late planes' tracking system would have warranted a multimillion-dollar investment.

Find the KRIs That Need to Be Reported to the Board

I have already alluded to the fact that the internal CSFs ascertained in the CSF workshop are the source for all performance measures. I now wish to clarify this. I once thought that the CSFs would also be the source of the KRIs. However, on a recent project it was made clear to me that the board tends to look at the organization from the outside in and thus, to them, the CSFs must reflect the progress with regard to strategy.

To ascertain the KRIs one needs to look at the external issues that concern the board and the strategy objectives, and from these objectives the KRIs will emerge. In many cases the KRIs have already been described in graphs to the board and incorrectly been called KPIs. We need to take these board measures and rebrand them as KRIs.

The importance of separating the KRIs from the KPIs should not be underestimated. KRIs will paint the picture of how the organization is performing and help keep the board focused on strategic issues.

Key Tasks for Finding the KRIs That Need to Be Reported to the Board

The key tasks for ascertaining the possible KRIs include these tasks:

Task 1. Review the strategic objectives and see what measures best describe progress

Commonly these measures will have already been graphed in strategic documents or previous board papers. Gather these graphs and, now using Stephen Few's guidelines, improve the presentation of the data.

Task 2. Review the operational critical success factors

As mentioned earlier, the internal critical success factors will not be the sole driver for the KRIs as the focus on inputs and activities; also to be considered are the outcomes and impacts which interest the board. However, the CSFs will help give some direction as to what are some good KRIs (e.g., the CSF timely departure and arrival of planes would stimulate a graph to show the timeliness of planes across the main regions of the world).

Task 3. Limit the KRIs to no more than 10

At first you may have up to 25 KRIs. Although there is no magic number, few organizations will need more than 10 KRIs. So start a process to weed out unnecessary KRIs. This can be done in discussion with the SMT and the board members.

Task 4. Ensure that the KRIs impact all of the six BSC perspectives

We need to ensure the board dashboard is balanced. Thus, the KRIs should be mapped against the six balanced scorecard perspectives discussed earlier in this book.

Task 5. Discuss the KRIs with a board member before presenting the KRIs to the board

It is important to presell the KRIs, so select one member of the board who is influential and seek their input before the board meeting. The aim is to get their buy-in and ensure that they are vocal in the board meeting. Ideally they should speak immediately after your presentation stating their support.

Find the Winning KPIs

It is recommended that the selection of organizational KPIs be started after the staff KPI workshops have been held and teams have made some progress with their own measures at the team level (level 4 in Exhibit 3.12).

No matter how complex your organization—whether a public body, a hospital, or a diverse manufacturer—team, department, and division performance measures should not be consolidated to become the organizational measures. Doing this creates chaos (e.g., some hospitals have over 200 measures at the organizational level). It is crucial that all staff members fully understand KPIs (see Chapter 1). Remember, finding appropriate KPIs is very much like peeling the layers off an onion to get to the core. Although it is relatively easy to produce a reasonable list of performance indicators, it is difficult to identify the key performance indicators, particularly when it is remembered that there will be fewer than 10 in the entire organization.

Key Tasks for Finding the Winning KPIs

The key tasks for ascertaining the possible winning KPIs include these tasks:

Task 1. Ensure That All KPIs Have All of the KPI Characteristics

Ensure that all KPIs selected pass this checklist. See Chapter 4 for the layout for the form. The characteristics of a KPI are:

  • Nonfinancial measures (that is, not expressed in dollars, yen, euros, etc.)
  • Measured frequently (e.g., 24/7, daily, or weekly)
  • Acted on by CEO and senior management team
  • Clear indications of what action is required by staff (e.g., staff can understand the measures and know what to fix)
  • Measures that tie responsibility down to a team or a group of teams that work closely together
  • Significant impact (e.g., it impacts most of the core CSFs and more than one BSC perspective)
  • Encouragement for appropriate action (e.g., have been tested to ensure they have a positive impact on performance, a minimal dark side)

Task 2. Limit the Organization-Wide KPIs to No More Than 10

There is no magic number, but few organizations will need more than 10 KPIs, and in fact many can operate successfully on fewer than five KPIs.

Task 3. Test All the KPIs in Three Pilots

It is important to test the KPIs to ensure they are working as expected and are having the desired behavioral outcomes.

To assist I have provided a checklist in PDF format for you to download.

Measures Gallery

Hold a “Measures Gallery” session as suggested by Stacey Barr, which is an open session where staff members are invited to share their views on the measures that have been displayed on the walls of the room. Visitors are encouraged to give feedback by writing their thoughts on Post-it stickers, which then are placed on the wall by the appropriate measure. This technique has been used successfully by a number of Stacey Barr's clients.

I like the concept as it is a way of accessing the “wisdom of the crowd.” James Surowiecki4 wrote that “a large group of people are often smarter than the smartest people in them.” Hence the term “wisdom of the crowd” was born. In other words, a group's aggregated answers to questions that involve quantity estimation have generally been found to be as good as, and often better than, the answer given by any of the individuals in the group.

It also gives another form of testing of the dark side. Staff can put on a sticker, “The last time we used this measure in 200x the following dysfunctional behavior resulted _________________”.

I imagine a measures gallery would look something like Exhibit 12.7.

c12ex007

Exhibit 12.7 A Measures Gallery Used to Gather Feedback on Measures

Benefits of This Stage

Determining the measures that will work for the organization will have a profound impact on the organization, stimulating timely action and linking day-to-day activities to the strategic objectives of the organization.

In addition, this stage will improve job satisfaction (e.g., measures that increase the level of staff recognition), increase job security in the longer term as teams contribute more to the bottom line, and provide a basis for recognizing and celebrating team achievements.

Templates and Checklists

To assist the KPI project team on the journey, templates and checklists have been provided. The reader can access, free of charge, a PDF of the suggested worksheets, checklists, and templates from kpi.davidparmenter.com/thirdedition.

The templates include:

image

Notes