Let the flow manage the processes, and not let management manage the flow.
—Taiichi Ohno

3.3

OPERATIONS REVIEW

David Anderson introduced us to operations reviews at Corbis. Everyone on his management team had to present their team’s metrics. It was my first experience as a team lead presenting metrics to a group of more than thirty people. I was terrified standing in the conference room: my voice shook, my heart pounded, and I thought I was going to maybe throw up. What if my presentation let the team down?

This experience taught me that management responsibilities include knowing the demand on my team and being able to present what the demand looked like in relation to the capability of the team to meet the demand.

Figure 49 is a cumulative flow diagram (CFD)—a stacked line chart that shows the amount of WIP and items delivered over time. I presented this CFD monthly at the operations review on behalf of the Build and Release team. When June rolled around, incoming requests spiked, and because I was able to demonstrate this spike, my request for more headcount was approved.

Figure 48. Cumulative Flow Diagram for Ops Review

The goal of the recurring monthly operations reviews is to collectively look at the data to see and understand the health of the organization. A disciplined and consistent review of organizational health provides a great opportunity for continuous improvement. Operations reviews provide feedback loops to help you understand how the organization is doing and to enable you to make sound decisions concerning your next move. They are objective data-driven retrospectives on the organization’s performance.

The reviews are organization-wide and involve senior leaders, managers, leads, and individual contributors to communicate that the organization takes performance seriously by setting an expectation for objective, data-driven, quantitative management. In other words, operations reviews are one way in which we can make work more visible by seeing what has come before.

Here are some logistics for how to run a successful operations review: Each manager has five minutes to present their team’s metrics for the month. Allow two to three minutes for questions or comments from the audience. Time-boxing the presentations and the following question and answer session helps keep people focused on what’s important, gives speakers their fair share of time, and avoids going off track or consuming too much time. Time-boxing avoids the common problem that occurs when you give someone a mic and can’t get them off the stage.

Sample operations review agenda:

Metrics to Present

In order to understand how we are doing, what the risks are, and how to improve predictability, Ops reviews show how the teams actually performed the previous month against promises/expectations.

When beginning operations reviews, the suggestion is for each lead to report on the following metrics and data for the first few months:

  1. Throughput: Throughput is how many things were completed over a period of time. One way to show throughput is with a CFD. The CFD will also show the ratio of incoming requests against completed requests and the amount of WIP at each stage of the workflow.
  2. Flow time: Look at how long it took items to move across the board from the time work was pulled into an in-progress column through delivery. The visual (hopefully a report easily generated by your workflow tool) shows the actual flow time for each completed work item for the previous month. This is useful for learning so you can make changes to your processes, your system, or to decrease variability and increase predictability. Or, if the card size is similar in nature, we may find it interesting to calculate the average cycle time or lead time.
  3. Issues and blocked work items: Identify any major issues or blocked work that is preventing the team from making progress. This will help people understand why things took so long and know what changes are underway to prevent those problems from happening again.
  4. The Time Thief O’Gram: Select one or more time thieves to report on to expose their theft.

Additional metrics you may want to consider presenting:

Future Operations Reviews

For each metric, we want to track the trend over time so we can see the improvements (or not). If we are to demonstrate continuous improvement, we want the mean trend to improve over time. To demonstrate predictability, we want the spread of variation to decrease over time. For example, to demonstrate that one can predictably arrive to work on time, it helps when the frequency of late arrivals decreases over time and the length of the lateness shrinks.

Another example: Amtrak has a passenger train from Portland to Seattle that is supposed to run on a set schedule throughout the day. According to the schedule, the last train of the day is supposed to arrive in Seattle at 8:05 p.m. Sometimes the train arrives at 8:25 p.m. Other times it arrives at 2:30 a.m. So, the train is unpredictable. The wide spread in arrival time causes variation in the train system.

The variable arrival times are caused by several factors. For example, our rainy Pacific Northwest weather generates unpredictable landslides, blocking trains until the rails are cleared (Thief Unplanned Work). Furthermore, because freight trains have a higher priority than passenger trains, and there are only so many train tracks that run through the tunnel, the Amtrak train takes a backseat to the freight trains (Thief Conflicting Priorities).

In order for Amtrak to demonstrate predictability, they would need to address landslides and change prioritization policies to decrease the variation of arrival times of the train from Portland to Seattle.

These are the types of decisions that operations reviews make possible. Without good, objective metrics, it is very difficult to understand exactly how the various thieves steal away our time and energy. But by making work visible, we are able to see the patterns and communicate to our organizations just where the problems lie, so we can learn and adjust and improve.

KEY TAKEAWAYS