5 - Context is King

One of the goals of Lean practices is to eliminate waste. But as we’ve already discussed, the components and outputs of manufacturing are more visible than those of IT, meaning that in manufacturing, the identification of waste is somewhat easier than in the IT world. In manufacturing, we can physically see scrap material, we can see the products that are in storage, we can count the items currently in progress, and when a machine is idle, we can easily see that. In IT, we are dealing with nonphysical items, which makes some of this more difficult to do. And if an IT worker is idle, we don’t usually notice it, as she will find something to do. Or to say it with Parkinsons’ law, “Work expands so as to fill the time available for its completion.”

The best way to enable all our workers to eliminate wasteful activities or idle time in IT is for them to understand the context of their work. This will allow them to make their own decisions instead of waiting idly until a decision is being made somewhere else in the organization. It will also enable them to make the right decisions to avoid spending effort on activities that don’t lead to the right outcomes. This aligns very well with Dan Pink’s motivational factors, as context will allow your people to make autonomous decisions and understand the purpose of the work they are doing.

In our legacy-inspired organization, we believed in the manufacturing approach to delivery, where handovers between machines are not hindering the outcome as long as they are part of a well-defined process. A machine or assembly worker does not need to know why he screws a screw into the metal plate at exactly this position. There is no alternative solution to having the screw in the exact right spot, so that context is not as important.

In IT, context does matter, as coding a specific functionality only works if you understand the context in which the function will be used. I was a developer myself, and when you are limited by the technical design without context, you aren’t able to solve problems that arise. But if you understand the context, you can develop a solution to solve the problem even when the design is flawed.

When companies embark on projects, they spend time elaborating on the idea for the project, build a business case for it, and in many cases, spend a lot of time maturing it before it can be kicked off. A lot of context is being created, yet in many organizations, this context is then transferred to the project team purely as documentation. And to make things worse, the project team usually spins up over time so that testers who join later will only learn the context from outdated documentation and “games of telephone.”

Furthermore, it is important to note that with any handover, some context will always be lost. Unfortunately, no amount of comments and documentation will ever really provide the full context, which I experienced when getting code back that still did not solve the problem I encountered as functional tester and had documented in a defect record.

Yet passing on context information—even if some of it will be lost in the process—is crucial. People make decisions based on their context and incentive. It is rare that someone involved in a project makes a bad decision due to malevolence; it’s usually because he did what he thought to be right with the context he had. So, how can we create a shared context to make the right decisions?

I learned from many successful Agile projects/programs/initiatives that a discovery phase can solve this problem. Having such a discovery activity for a project or feature, or whatever the right unit of work is, provides an investment that pays back over time. I will use the example of a finite Agile project that runs over multiple releases, but you can adopt the same for any project size.

A discovery workshop that can take anywhere from a few hours to many weeks is best broken down into three separate phases:

The first part of discovery is to get the team and stakeholders on the same page. To do this, you have to gather together as many of the people who will deliver the solution involved as is feasible. For a small delivery team, this should actually be the whole team. I have run Agile initiatives that required several dozen people in the delivery teams; in that case, I would invite representation from all the different teams and skills involved (e.g., developers, business analysts, testers, people from the mainframe team, people from the Java team). You can easily run the session with around thirty people in the room. (SAFe provides mechanisms for even larger teams with the PI [program increment] planning ceremony.) The focus of the first part of discovery is to understand what the team is coming together to solve.

Given that almost all initiatives in an organization require a business case to get funding, the business case is usually a good starting point. The sponsor presents the business case to the team so that everyone knows what has been promised and how success will be measured. While each discovery is tailored to the specific context, here are a few of the many common activities that help to set this context and solidify understanding of the business problem:

Now that the team understands the context of the project, we can dive into finding a solution for the problem. Of course, you will have done some of this already when you created the business case (after all, you needed to identify who to involve in the discovery session). It is likely you have a good idea which applications will be impacted and you have a high-level solution in mind on which the business case was based.

This part of discovery can feel a bit chaotic, as it has a strong exploratory dimension to it. You want everyone to collaborate in finding the most appropriate solution to your problem. To do this, you want to do a bit of to-be process design and identify the technical architecture of the solution. Where required, you can do deep dives into parts of the process or into specific technologies. This is a great way to identify risks and validate assumptions about what a technology can do for you. Where residual concerns exist, you might already identify spikes that you want to run during delivery.

You will have to find the right balance for your organization between documenting this in detail and having no documentation at all; too much documentation wastes time and effort, while no documentation at all will make it difficult to guide the solution during the project. I recommend diagrams on whiteboards, on PowerPoint slides, or in Visio as a good starting point. High-level architecture should be something that fits on a whiteboard and is documented—and not just in people’s heads.

The process-design and technical-architecture workshops will have an iterative character as you keep revisiting them to make improvements as the solution evolves during discovery. You created a rich context in the first part of discovery, so keep referring to it to make sure you are focusing on the right things, respecting the scope constraint and everything else defined before. In some cases, you might decide to change outputs from the first part, which is okay too. Once you have done this a few times, you will be amazed by how often the outputs of the first part allow you to steer the discussion to a fruitful outcome. A little hint here: paste the outcomes of the first part on the wall so that they are always visible.

You will also want to start writing up the features of the solution which you might structure under related epics. This list can already be exhaustive or you can leave yourself some room for additional features to be identified. What you need to define in all cases is the minimum viable product (MVP). What is the minimum amount of scope that needs to be completed before you can go live? This will feature prominently later, when the release planning happens. The tendency is to make the MVP too large, so challenge yourself to keep culling scope until it is truly minimal. In large organizations with complex architecture, my guiding rule is that an MVP needs to be deliverable within three months. Your mileage might vary, though.

But wait, you say, shouldn’t Agile be flexible? Why are we defining the solution in discovery and not just letting it emerge through the iterations? In my experience from working in large organizations, I can tell you that you need some guardrails. I have seen projects get derailed for months when, after three iterations, stakeholders decided to move functionality from the CRM system to the ERP system. The team structures had to change and commercial constructs were impacted, as were test strategies and many more things. This extreme idea of flexibility is beyond what an initiative can absorb. Think about discovery as an outline of the Mona Lisa, that you will fill in with details during delivery. You want to avoid ending up with a picture of an abstract depiction of love when you were after a picture of a gorgeous lady.

Figure 5.1: Discovery versus delivery: Discovery is like the first outline of a picture; delivery fills in the details

After setting the context and finding a viable solution, the last part of discovery is about getting ready for delivery. This part is crucial and will probably take you a while to do, as it involves mobilizing teams, setting up your workspace, estimating the work ahead, and planning the release schedule. This part tends to require less involvement from the business stakeholders and is heavier on the IT delivery team.

Depending on where your organization is with regard to the adoption of Agile and DevOps, the first thing to align is the terminology, methodology, and practices you will use on the project. I personally prefer to use an industry-wide framework such as the Scaled Agile Framework (SAFe), as it provides common terminology and practices as an easy starting point. Of course, you can leverage your own methodology or define a new one. A couple of words of warning, though: If you choose to rename a term because you don’t like the sound of it (e.g., from Scrum master to delivery facilitator, which I had to do for a client), then you lose the ability to get guidance from the community. After all, you cannot Google that term, and other people outside your company will not understand you either. The other thing that I am very passionate about is a common structure for your scope hierarchy. I can show you the scars of what happens when you don’t have that. If you use

in the same organization, it will, at some stage, lead to confusion. This is flexibility for the sake of flexibility at the cost of organizational effectiveness. Eventually, these terms will collide, so just choose one and stick to it. After all, it’s just terminology; consistency is much more important than personal preference. You might also want to run training sessions for the team, the product owner, and any other key roles during this time.

Especially in an organization with a lot of legacy applications, the technical ecosystem becomes important. Environments are usually limited and you have to orchestrate delivery with other parts of the organization, so understanding what is available to you becomes a critical element for success.

This is also the right time to do the initial estimation of the project scope, for which you can use planning poker or other Agile estimation practices to start preparing the backlog. You will need to “prime the pump” so that there is enough defined scope for the first few iterations that complies with the “definition of ready” for the team to be successful and effective. There will be ongoing backlog grooming, but in my experience, it takes a while before the grooming can keep up with delivery. I’ve had many teams run out of scope during delivery; then the team becomes distracted and inefficient as they take in unready scope, search for scope down the backlog, or—worse still—spin their wheels for a while. Having two to three iterations’ worth of scope ready to go usually provides enough runway for the team to get the backlog grooming to the right speed.

Discovery ends with two large activities that require preparation, the discovery showcase and the planning event (or PI planning if you follow SAFe). The showcase shares the outcomes of discovery with the rest of the organization. It is usually a one- to two-hour session highlighting the outputs from discovery. The planning event brings together the delivery team and stakeholders to plan the first few months. The focus here is on dependencies between teams, having a baseline of goals per iteration, and release and joint-risk management. (The guidelines for PI planning on the SAFe site are quite good.) You want to make sure that your organization is ready across both business and IT. You want to make sure that the content for the PI planning has been prepared and that you have the right facilities to conduct the session, especially when technology is required to communicate across multiple locations.1 Some people might also be familiar with the term “big room planning” from Lean, which achieves the same by having all teams plan together in a large room, forcing collaboration across teams and frequent communication across organizational boundaries.2

Once you have gone through discovery, the team that will deliver the solution has a rich context to work within. It will allow the team and each individual to make decisions that are more likely to be correct. Thinking back to the Dan Pink means for motivation, we have provided autonomy by allowing the team to be part of the decision-making process during discovery (especially during planning and estimation, with which they are less involved in traditional approaches); we gave them purpose by interacting with the business stakeholders and spending time understanding the business problem and impact on customers. Context is also important in regard to the last aspect: mastery.

Mastery requires context and understanding of the area that we are trying to master. One challenge to achieve mastery is a cognitive bias called the Dunning–Kruger effect, which is sometimes also called “illusionary superiority effect.”3 The Dunning–Kruger effect occurs when you don’t know what good looks like (e.g., have the right context for your assessment) but you think you are pretty good at it, which explains why we so very often declare victory early. Look at all the organizations that seem to evaluate agility by how many Post-it notes are visible in their offices. I often encounter this when doing maturity assessments, where the teams with the lowest maturity claim to do practices that they don’t fully understand, and where the high-maturity teams see a lot of possible improvements and rate themselves low in maturity.

I have learned about Dunning–Kruger the hard way. As a consultant, I am frequently asked to do maturity assessments. Given you usually have to rely on asking questions, you can easily be encountering Dunning–Kruger. Here is how that would go:

Me: [Thinking, Okay, I could check the yes box, but let’s be sure.] How do you know you are using continuous integration?

Me: [Phew!] Okay, there is some more education to be done before this assessment makes sense.

Together with one of my clients, we realized that a self-assessment would not provide us with the right results due to the Dunning–Kruger effect and came up with an alternative approach. Taking a lesson from my university days, when I spent too much time playing a computer game called Civilization, we used the idea of a technology tree, which provides guidance for the different technologies required in order to research the next one (e.g., you need astronomy before navigation becomes reliable).4 In our continuous delivery technology tree, we provided context to the teams that described the dependencies between practices (for example, that continuous integration required automated builds—builds triggered from check-in and automated unit tests). This, in turn, allowed teams to assess themselves in the context of the full technology tree, making it easier for them to chart their way to mastery across the tree.

First Steps for Your Organization

Run a Discovery Workshop for an Initiative of Your Choice

I described the discovery workshop in some detail in this chapter, and I encourage you to run a session in your organization. Here is a sample agenda for a two-week long discovery workshop, which you can adjust as required for larger or smaller initiatives. The activities highlighted are just an example; there are many additional activities that you can embed in your discovery sessions to enhance the experience.

Part 1: Explore the Business Problem (Two Days)

briefing by initiative sponsor (leveraging the business case where possible)
current customer/stakeholder experience
to-be customer experience workshop
creating the mission statement
success criteria definition
in-scope and out-of-scope brainstorming
stakeholder management workshop
prioritization-scheme discussion
risks, issues, and dependencies identification and mitigation

Part 2: Solutioning (Three Days)

high-level to-be process design
high-level technical architecture to support to-be design
breaking the scope up into features
identification of minimum viable product (MVP)
selected deep dives into technologies and processes

Part 3: Planning for Delivery (Five Days)

Agile training (if required)
team structure definition
estimation of scope
high-level roadmapping
technical preparation (environment, quality, configuration management strategies)
setting up delivery governance
social contract
prepping the backlog
discovery showcase to the organization
planning workshop (PI planning when using SAFe)