Inspect and Adapt

Kaizen is about changing the way things are. If you assume that things are all right the way they are, you can’t do kaizen. So change something!

—Taiichi Ohno

The Inspect and Adapt (I&A) workshop is a significant event, held at the end of each Program Increment (PI), in which the current state of the Solution is demonstrated and evaluated by the train. Teams then reflect and identify improvement backlog items via a structured, problem-solving workshop.

One statement from the Agile Manifesto summarizes how important the philosophy of continuous improvement is to the SAFe Lean-Agile approach: “At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.”

SAFe emphasizes the importance of this philosophy by taking it a step further—that is, by including relentless improvement as one of the four pillars of the SAFe House of Lean. While opportunities to improve can and should occur continuously (e.g., Iteration Retrospectives), applying some structure, cadence, and synchronization helps ensure that time is set aside to consider what can be done better during the Agile Release Train (ART) I&A event.

Details

All program stakeholders participate with the Agile Teams in the ART I&A workshop. The result is a set of improvement backlog items that the teams add to the backlog for the next PI Planning event. In this way, every ART improves every PI. For large solutions, a similar I&A workshop also occurs for the Solution Train.

The I&A workshop consists of three parts:

1. PI System Demo

2. Quantitative measurement

3. Retrospective and problem-solving workshop

Participants in the ART I&A workshop should include, wherever possible, all the people involved in building the system:

• The Agile Teams

• Release Train Engineer (RTE)

• System and Solution Architect/Engineering

• Product Management, Business Owners, and others on the train

Additionally, Value Stream stakeholders may attend this workshop.

PI System Demo

The PI System Demo is the first part of the ART I&A workshop. This demo is a little different from the biweekly ones that precede it, in that it is intended to show all the Features that the ART has developed over the course of the PI. Typically, the audience is broader; for example, additional Customer representatives may choose to attend this demo. Therefore, the PI system demo tends to be a little more formal, and some extra preparation and staging are usually required. Like any other system demo, though, it should be timeboxed to an hour or less, with the level of abstraction remaining high enough to keep stakeholders actively engaged and providing feedback.

During the PI system demo, Business Owners, customers, and other vital stakeholders collaborate with each Agile team to rate their actual business value achieved as shown in Figure 1.

The PI objectives are as follows: structured locations and validation of locations, build and demonstrate a proof of concept for context images, implement negative triangulation by tags, companies, and people, speed up indexing by 50 percent, index 1.2 billion more web pages, and extract and build URL abstracts. The stretch objectives are as follows: fuzzy search by full name and improve tag quality to 80 percent relevance. Business value shows two columns: plan and actual. The plan column reads, 7, 8, 8, 10, 10, 7, 7, and 4. The actual column reads, 7, 8, 6, 5, 8, 7, 0, and 4. The section, totals at the bottom reads percent achievement (90 percent), 40, and 45.

Figure 1. Business Owners collaborate with each team to rate their PI objectives during the PI system demo

Quantitative Measurement

In the second part of the workshop, teams review any quantitative Metrics they have agreed to collect and then discuss these data and any trends. In preparation for this part of the workshop, the RTE and the Solution Train Engineer are often responsible for gathering the necessary information, analyzing it to identify potential issues, and facilitating the presentation of the findings to the ART.

One primary measurement is the program predictability measure. Each team’s planned versus actual business value is rolled up to the program level in the program predictability measure, as shown in Figure 2.

A stack of three overlapping rectangles represent the team PI performance reports. The first rectangle shows the PI objectives along with business value. A graph titled “program predictability measure” is shown. The vertical axis of the graph labeled “program objectives achieved” ranges from 0 to 120, in increments of 20. The horizontal axis is marked as follows: PI 1, PI 2, PI 3, PI 4, and PI 5. The graph shows three curves representing team A: out-of-control development, team B: controlled development, and program (ART). A text box on the graph reads, target effective process control range, predictability sufficient to run the business, handles common variations, and special causes may still cause excess variation. An arrow from the team PI performance reports points to program predictability measure.

Figure 2. The program predictability measure is rolled up from each team’s planned versus actual business value

Reliable trains should operate in the 80–100 percent range; this allows the business and its outside stakeholders to plan effectively. Note that stretch objectives don’t count toward the commitment but do count toward the actual score, as can also be seen in Figure 1.

Retrospective

The teams then run a brief (30 minutes or less) retrospective, whose goal is to identify whatever issues they would like to address. There is no one way to do this; several different Agile retrospective formats can be used [3]. The objective of the retrospective is to identify a few significant problems that the teams can potentially address.

Based on the attendees present at the retrospective and the nature of the problems identified, the facilitator helps the group decide which issues they want to tackle. Each team then has a choice of resolving Team Level problems or, more typically, selecting a program-level problem and joining others who wish to work on the same issue. This self-selection helps provide cross-functional and differing views of the problem, and it seeds the problem-solving workshop with those who are most likely to be impacted and those who are best motivated to address the issue.

Key ART stakeholders—including Business Owners, customers, and management—join the teams in the retrospective and problem-solving workshop that follow. Often the Business Owners, acting alone, can unblock the impediments that exist outside the team’s control.

Problem-Solving Workshop

To address program-level problems, the ART holds a structured, root cause problem-solving workshop. Root cause analysis provides a set of problem-solving tools that can be used to identify the actual causes of a problem, rather than just addressing the symptoms. This session is typically facilitated by the RTE, or another facilitator, in a timebox of two hours or less. Figure 3 illustrates the steps in the problem-solving workshop and the following sections describe each step in more detail.

Six rectangles titled, agree on the problem to solve, apply root cause analysis (+ five whys), identify the biggest root cause using Pareto analysis, restate the new problem for the biggest root cause, brainstorm solutions, and identify improvement backlog items are shown. The first rectangle reads, insufficiently reliable release commitments. The second rectangle shows a rightward arrow. Two blocks labeled “people” and “process” points to the top of the arrow. Three blocks labeled “tools,” “program,” and “environment” points to the bottom of the arrow. An arrow labeled “cause of cause of cause of cause of cause 1” points to another arrow labeled “cause of cause of cause of cause 1,” which in turn points to an arrow labeled “cause of cause of cause 1.” Further, points to an arrow labeled “cause of cause 1,” in turn points to an arrow labeled “cause 1.” The arrow labeled “cause 1” points to people. A warning box reads, insufficiently reliable release commitments. The third rectangle shows a chart. The horizontal axis represents the causes. The vertical axis represents the votes and it ranges from 0 to 9, in increments of 1. The graph infers the following data: cause 1, 8; cause 2, 3; cause 3, 3; cause 4, 2; cause 5, 1; and cause 6, 0. The fourth rectangle reads, insufficient architectural runway. The sixth rectangle shows a stack of three layers with an oval labeled “NFRs” attached to the base of the stack.

Figure 3. Problem-solving workshop format

Agree on the Problem(s) to Solve

American inventor Charles Kettering is credited with the statement that “a problem well stated is a problem half solved.” At this point, the teams have self-selected the problem they want to address. But do they agree on what the problem is, or (more likely) do they have differing perspectives? To clarify their understanding, the teams should spend a few minutes stating the problem, thinking about the what, where, when, and impact as succinctly as they can. Figure 4 illustrates a Baja Ride systems engineering example.

The first statement reads, we discovered three significant design problems in the October deployment of the new EMV vehicles at the Thrills Amusement Park. Three arrows labeled “what,” “when,” and “where” points to the first statement. The second statement reads, the design flaws caused us to recall the vehicles and invest three months in materials, redesign, and testing. We delivered late, paid substantial penalties, and lost credibility with the customer. An arrow labeled “impact” points to the second statement. Text at the bottom reads, concept contributed by Beth Miller.

Figure 4. Example problem statement

Perform Root Cause Analysis

Effective problem-solving tools include the fishbone diagram and the ‘Five Whys.’ Also known as an Ishikawa Diagram, a fishbone diagram is a visual tool that is used to explore the causes of specific events or sources of variation in a process. Figure 5 illustrates the fishbone diagram, in which the problem statement is written at the head of the ‘fish.’

The diagram shows a rightward arrow. Two blocks labeled “people” and “process” points to the top of the arrow. Three blocks labeled “tools,” “program,” and “environment” points to the bottom of the arrow. An arrow labeled “cause of cause of cause 1” points to another arrow labeled “cause of cause 1,” which in turn points to an arrow labeled “cause 1.” The arrow labeled “cause 1” points to people. The rightward arrow points to a cross mark box that reads, insufficiently reliable release commitments.

Figure 5. Fishbone diagram with major sources identified

In the problem-solving workshop, this diagram is preloaded with the main bones—that is, the categories of people, process, tools, program, and environment. Causes are identified and then grouped into categories as bones off the main bone. However, these may be adapted as appropriate.

Team members then brainstorm factors that they think contribute to the problem to be solved. Once a root cause is identified, its cause is identified with the ‘Five Whys’ technique. By simply asking why as many as five times, each cause-of-a-cause becomes easier to discover and can be added to the diagram.

Identify the Biggest Root Cause

Pareto Analysis, also known as the 80/20 rule, is a technique used to narrow down the number of actions that produce the most significant overall effect. It uses the principle that 20 percent of the causes are responsible for 80 percent of the problem. Pareto analysis is especially useful when many possible courses of action are competing for attention, which is almost always the case with complex, systemic issues.

Once all the possible causes-of-causes have been identified, team members cumulatively vote on the item they think is the most significant factor causing the end problem. They can do this by placing stars (five stars are allocated to each group member, which can be spread among one or more items as they see fit) on the causes they think are most problematic. The team then summarizes the votes in a Pareto chart (as shown in Figure 6), which illustrates their collective consensus on the most significant root cause.

The horizontal axis represents the causes and the vertical axis represents the votes. The vertical axis ranges from 0 to 9, in increments of 1. The graph infers the following data: Cause 1, 8; Cause 2, 3; Cause 3, 3; Cause 4, 2; Cause 5, 1; and Cause 6, 0.

Figure 6. Pareto chart of probable causes

Restate the New Problem

The next step is to pick the largest cause from the list and restate it clearly as a problem. This should take only a few minutes or so, as the teams are very close the root cause now.

Brainstorm Solutions

At this point, the root cause will start to imply some potential solutions. The working group brainstorms as many possible corrective actions as they can think of within a fixed timebox (about 15–30 minutes). The rules of brainstorming apply here:

• Generate as many ideas as possible.

• Do not allow criticism or debate.

• Let the imagination soar.

• Explore and combine ideas.

Create Improvement Backlog Items

The team then cumulatively votes on up to three most likely solutions. These solutions will serve as improvement stories and features to be fed directly into the PI planning session that follows. During that session, the RTE helps ensure that the relevant improvement stories are loaded onto the iteration plans, thereby ensuring that action will be taken and resources allocated to them, as with any other backlog item. This closes the loop on the retrospective and ensures that people and resources are dedicated as necessary to improve the current state.

In this way, problem-solving becomes routine and systematic at both the program and large solution levels. Team members, program stakeholders, and large solution stakeholders can be assured that the value stream is solidly launched on its journey of relentless improvement.

Inspect and Adapt for the Solution Train

The preceding sections described a rigorous approach to problem-solving in the context of a single ART. The ART I&A often includes key stakeholders from the Solution Train, which is the recommended path to facilitate solution development. In larger value streams, however, an additional Solution Train I&A workshop may be required, following the same format.

Due to the number of people in a Solution Train, not everyone can attend the Solution Train I&A workshop. Instead, those stakeholders who are best suited to address that context are invited to participate. This group includes the primary stakeholders of the Solution Train, as well as representatives from the ARTs and Suppliers.

LEARN MORE

[1] Leffingwell, Dean. Agile Software Requirements: Lean Requirements Practices for Teams, Programs, and the Enterprise. Addison-Wesley, 2011.

[2] Leffingwell, Dean. Scaling Software Agility: Best Practices for Large Enterprises. Addison-Wesley, 2007.

[3] Derby, Esther, and Diana Larsen. Agile Retrospectives: Making Good Teams Great. Pragmatic Bookshelf, 2006.