Sprint Review Bazaar in LeSS
LeSS Sprint Review & Retrospectives
• Guide: Adapt the Product Early and Often
• Guide: Overall Retrospective
• Guide: Multi-Area Reviews & Retrospectives
Sprint Review Bazaar in LeSS
A constitution should be short and obscure.
—Napoléon Bonaparte
At the heart of Scrum is empirical process control for both the product and how it’s created. Create a small shippable slice of the product, then inspect what and how, and adapt both. In essence, that’s the purpose of the Sprint Review and Retrospective.
In the Sprint Review the users/customers and other stakeholders learn with the Product Owner and Team. Users hands-on explore the new items. Everyone explores what’s going on in the market and with users. And last but not least, they discuss what to do in the future. In the Sprint Retrospective the Team reflects on their experience and explores how to easily deliver an amazing product increment that improves the environment, and makes lives better—including their lives. They create an experiment to try in the next Sprint, aiming towards this impossible perfection.
The upcoming guides cover Review and Overall Retrospective, but not single-team Retrospectives. Related principles when scaling:
Customer-centric—“Why would we want to include users/customers in every Sprint Review?” Old groups aren’t used to learning together across silos. We’ve met far too many teams that never met a user and were afraid to include users in reviews, as that would mean real transparency.
Transparency—Executives espouse the salutary benefit of transparency, but watch what happens when painfully true transparency is enabled in a group new to LeSS. Ouch! Many groups are opaque and afraid to reveal the actual messy state of affairs. It’s hard to get over that.
Continuous improvement towards perfection—We’ve had clients that once a year held a postmortem to create heavenly improvements for next year’s fantasy. And worked in many big groups that said “Things are basically good enough.” There’s no intrinsic desire for improvement.
Empirical process control—Many large-scale organizations have a centralized process or PMO group tasked with improvement, based on a Taylorist culture of pushing “improvements” onto teams. There’s no sense of empowerment or engagement. The notion of empirical process control for the product and how it’s created every Sprint is a million miles away from their habits.
Whole-product focus and Systems thinking—Big groups with silo teams don’t have the attitude and behavior of looking at the whole, being responsible for the whole, and thinking about the system.
If your entire company is nine people, we hope you won’t do something as dumb as creating an annual plan of scope and schedule, and trying to march on-plan towards a big-batch end-date for user acceptance testing. But the bigger the product group, the more likely that institutionalized dumbness exists, for reasons packed in a can of worms that we won’t open. The upshot is that when a large group transitions to LeSS, there’s a chance they’ll carry the baggage of predictive planning and inspection for acceptance into the Sprint Review, which becomes an event to see if the group is on schedule and if the items will be accepted.
Don’t do that. Instead, try agility and learning together. In the Sprint Review seek new information about profit drivers, strategic customers, business risks, competitors, new problems and opportunities, in order to adapt and decide the product direction for next Sprint. And discuss together about the new items—everyone learning something. Repeat forever. This is a major mindset and behavior change for large groups.
A Sprint Review bazaar is analogous to a science fair: A large room has multiple areas, each staffed by team representatives, where the items developed are explored and discussed together with users, teams, etc. The opening photo in this chapter shows an example.
Note! The bazaar is not the whole Review. There’s also the critically important post-bazaar discussing and deciding what to do next.
For a bazaar, the macro-level steps for the Sprint Review are (1) diverge for a bazaar-style exploration of items, and (2) merge for all-together discussion with the Product Owner towards next steps. Reserve lots of time for the critical second step.
Example bazaar-phase steps:
1. Prepare different areas for exploring different sets of items. Include devices that are running the product. Team members are at each area to discuss with users, people from other teams, and other stakeholders. Learning happens both ways! Include paper feedback cards to record noteworthy points and questions.
2. Invite people—including other team members—to visit any area.
3. Start a short-duration timer (e.g. 15 minutes) during the exploration. The timer creates a cadence move-on to another area.
4. While people hands-on explore the items and discuss together, record noteworthy points on cards.
Tip: Avoid demos, as they don’t engage users and don’t provoke deep feedback. Rather, encourage hands-on use by users. Team members can answer questions or guide.
5. At the end of the short cycle, invite people to rotate or remain for another cycle. These mini-cycles help a more broad and diverse exploration of all items.
After the bazaar come the major all-together discussion steps:
1. People sort their feedback and question cards so that the Product Owner sees important ones first.
2. While all together, the Product Owner leads discussion of the feedback cards, as shown in Figure 14.2.
3. The Product Owner leads discussion on the market and customers, upcoming business, market feedback about the product, and broadly, what’s going on outside.
4. Most importantly for the entire Review, there’s a discussion—and perhaps a decision—about the direction for the next Sprint.
Multi-site—How to hold a bazaar when multiple sites are involved? One approach is to replicate it at each site (time zones permitting), but ensure that all feedback and questions get to the Product Owner. For the all-together discussions after the bazaar, try video conferencing tools.
Another or complementary option for the bazaar is for people to play with features on devices wherever they are. To record feedback, rather than using cards, consider digital tools such as a chat window for each item.
“We can’t do continuous delivery because of the deployment policies.” “We have too many sites.” “Our code is crap.” “Requests from government regulators take ages before we hear about them.” “We’re going too slow.” “The users aren’t participating.” “HR won’t let us.” “The vendors aren’t involved.”
These, and many more, are statements we’ve heard over the years in groups adopting LeSS. One thing they all have in common is they relate to all-team concerns and/or the overall system—spanning everyone and everything from concept to cash.
The time and place to deal with these systemic concerns—and to think about improving the system towards perfection—is the Overall Retrospective. Who’s there? The Product Owner, team representatives, Scrum Masters, and managers. Why? They’re all part of the system, with interest in improving it. They (1) discuss and learn about some aspects of the system, (2) create a systemic improvement experiment for the next Sprint, and (3) reflect on the results of the last retrospective experiment and use that to learn and adapt.
One of the LeSS principles is continuous improvement towards perfection. We once visited a huge group that was considering a LeSS adoption and a manager said, “We’re making a profit and have a stable customer base. Why should we bother to improve?” Ow! We’ve learned that dealing with this attitude is one of the harder challenges with early adoptions, because in the prior system so many people have been so disconnected from the customers and business results. Connecting teams with real customers and users, and engaging them in product ownership, are key steps to cultivating an intrinsic desire to improve towards perfection. And what is that? Well, there isn’t one answer, but there are examples:
The product is awesomely popular and profitable, defect-free, and features are created with ease.
The organization has agility; it can easily change direction with almost no friction or cost—it can turn on a dime for a dime.
Everyone has great breadth and depth of knowledge, cares deeply about the customers and product, and are happy in the work.
That should keep the group improving for awhile!
Some Overall Retrospective tips:
Reflect on the results of the last experiment.
As emphasized in the following guide, focus on the system.
Hold the Overall Retrospective early in the next Sprint, since the last day of the Sprint includes the Review and Team Retrospective meetings, and so people can be bored or burned out for meetings on that day.
Include at least two major steps: (1) the analysis of something systemic, and (2) the design of a systemic improvement experiment.
Create only one new experiment; be focused, and follow through.
Remember that, especially in large-scale systems, an experiment can involve many weeks or months of support and activity, so the new experiment may be strongly related to a prior one.
Multi-site—Try a multi-site Overall Retrospective with video and diverge–merge cycles. For example, (1) each site separately does 5-Whys analysis or systems modeling for a concern, (2) each site shares results, (3) sites separately brainstorm countermeasures, (4) sites together share these and pick an experiment. Also, as some issues are site specific (e.g. environment and culture), try site-level retrospectives. Figure 14.3 shows an example.
Multi-team Retrospectives—A Retrospective involving all team members from two or more Teams is another option in LeSS. Some teams might want to do that when, for example, they’ve been working closely together. But this doesn’t replace an Overall Retrospective, where the focus is on the system.
It’s instinctive for all of us to fall into the thinking mistake of local concerns and optimization. In an Overall Retrospective signs of that include—perhaps this will be a surprise—collecting results of the Team-level Retrospectives as the starting point of “overall” analysis. But this bottom-up approach misses an important insight of systems thinking: the system is not the sum of its parts. So beware of bottom-up. Of course, this doesn’t literally mean ignoring an important escalated issue coming from all the teams. Those need attention. We mean something subtler:
What’s the system? Everyone and everything from concept to cash, and all its dynamics in time and space. People, organizational design, physical and technical environments, and more are part of the system—and all related and interacting.
The first step of systems thinking is “simply” recognizing that there is a whole system, with elements that influence one another within a whole. These influences can have delays, create reinforcing cycles, and have unintended or hidden consequences, with a cascade of new influences.
In a way, “recognize there’s a system” seems a trivial idea without use. But that’s not true, because we Homo sapiens haven’t evolved brains for “What’s the nonlinear delay dynamics in our organization?” We evolved for “I’ll have chocolate, now.” And this local perspective is reinforced in big old organizations with single-specialized groups, as this causes the loss of system perspective. The Business Analysis group is concerned with their tasks and being locally efficient, and they don’t know—nor are expected to know—other perspectives. In short, there’s biology, structure, culture, and conditioning to see the part, not to see the whole.
Understand—How to apply systems thinking? How to understand the system, or more correctly, how to discuss and think about a model of the system? With a systems model, also known as a causal loop diagram. Now, superficially a systems model uses a specific visual-modeling language or notation, but first let’s step back and consider what’s going on in this session:
At a surface level they’re drawing a diagram in some notation, though that’s not very important. But the content and focus is. They’re thinking about and discussing the system and its dynamics, they are system thinking. Furthermore—and not to be underestimated—they are demonstrating the credo of good modeling:
While the group is in an Overall Retrospective sketching a systems model together, they’re exploring each other’s understanding of the as-is system and their beliefs. They’re taking complex and invisible notions in each other’s minds, and making them visible... “Oh! Now I see what you are thinking about the current system. Is that true?”
Understand More During Early Adoption—This guide has emphasized using systems modeling during retrospectives, but it’s also useful during “Step 0: Educate Everyone” when getting started with a LeSS adoption.
Action—The Overall Retrospective includes a second major step to design a systemic improvement experiment: action! Systems modeling can be used in this step too. For example, the group can speculate about a future to-be system model and discuss and explore its consequences. And they can discuss and model the dynamics of introducing a particular change experiment into the as-is system. What might happen? We can’t predict the future, but we can think about its scenarios. Besides the obvious action experiment, notice that something subtler can and will happen: people’s minds have changed as they learn a better model of the system, and that itself can “organically” lead to improved behaviors or decisions in the future, unrelated to any specific action.
There’s a non-trivial language for systems modeling, since systems are non-trivial. But the basics aren’t complicated, and good enough for lots of useful discussion. The basics:
variable—A thing expressed as a measurable quantity, such as velocity (rate of delivery) of features, and code quality.
causal link—Influence between variables, such as saying if number of features increases then waste increases, and vice versa.
Note! Thinking about interactions and causal relationships is the critical focus needed in systems thinking. And doubly so for large-scale systems, because the time and space are vast, and the interaction dynamics between the myriad parties are usually full of hidden but crucial facts and forces.
opposite effect—A causal link can have an opposite effect, such as if the percentage of weak developers goes up, then code quality goes down, and vice versa
The sketch in Figure 14.4 shows notation for variables, and direct and opposite causal links.
Tip: Sketch a systems model on a whiteboard with sticky notes for the variables—to make moving them easy.
Some other useful concepts and related notation:
delay—One key reason for flawed beliefs of how a system behaves is that influences can have delays. Causes and effects are not close in time—nor close in space in large-scale development. And delayed consequences, such as information loss, can be hidden in the interaction effects between groups. So people have trouble seeing and learning these dynamics. For example, managers are pushed to increase velocity, and do the quick fix of hiring many lowcost (and it turns out in this case, weak) developers. In the short run this quick fix gives the appearance of increasing velocity. But there is a long-term delayed consequence of reduced code quality, leading to a slower velocity eleven months later.
belief—Another key practice in systems modeling is to discuss beliefs. It’s one thing to sketch, claim, imply, or assume “managers can evaluate developers without looking in depth at their code,” but it’s another to recognize that could be belief, not fact. We model to have a conversation, so systems modeling is a time to discuss and become conscious of our beliefs, make them visible, and critique them.
Almost every causal link or variable is an opportunity to examine and discuss beliefs. Is Velocity a good variable to include? What does measuring it lead to? Do weak developers create bad code? What do you mean by “more features means more waste?”
The sketch in Figure 14.5 includes notation for delays, which are visualized with double lines through a causal link, and some informal notes capturing discussions. Of course, the notation doesn’t really matter as long as the group has a common understanding.
This example model isn’t meant to be “insightful,” it’s meant to illustrate that we model to have a conversation!
Learn—Learn more about systems thinking and modeling here:
Systems Thinking chapter in the first LeSS book, Scaling Lean & Agile Development: Thinking & Organizational Tools for Large-Scale Scrum, by Larman & Vodde. This chapter is also online at less.works.
The Fifth Discipline by Senge, an important classic.
Thinking in Systems, by Meadows & Wright
Systemantics, by Gall.
There are no LeSS Huge rules for reviews and retrospectives. The catch-all statement “All Sprint LeSS rules apply for each Requirement Area” implies a Sprint Review and Overall Retrospective for each separate Requirement Area. But there’s no requirement for meetings that span the entire product.
Review—Although not required, a multi-area Review spanning from two up to all areas (full product) is certainly possible when the group feels the need.
Why isn’t a product-level Sprint Review mandated in LeSS Huge? After all, avoiding one can reduce focusing on and seeing the whole. First, a group can hold a product-level review. But each area is often different enough that no great insights come from one—at least not every Sprint. And especially at Huge scale, in addition to gathering the entire Product Owner Team and representatives from many teams, the complexity of a product-level review could involve people at 10 sites worldwide, so can be a real pain to set up and run. There needs to be a compelling reason, and if there is, it’s probably not compelling for every Sprint.
Retrospective—Similarly, there’s no rule requiring a product-level Retrospective, but there may indeed be a good reason to hold a multi-area Retrospective since improving the system is key, and the organizational system spans the areas. A multi-area Retrospective is more likely when areas aren’t working well together, or several areas have similar problems. It’s also useful for teams from different Requirement Areas working together at the same physical site, to improve relationships and knowledge sharing.
