It was 8:30 a.m. on Friday morning in San Francisco, the final day of our sprint with Slack. Michael had scheduled our first customer interview for 9 a.m., and the sprint team was trickling in, coffee cups in hand. We rearranged the couches and chairs so everyone could face the video screen at the front of the room. Braden connected a laptop to the screen, opened a web browser, and joined a video conference that Michael had set up.
Slack started the week with a big challenge: their software was hard to explain to potential customers. Many of the benefits of using Slack (better communication, better teamwork, less stress about work) were only apparent once customers took a chance and tried it with their teams. Experimenting with a new piece of software is a lot of work, so Slack had to make the value clear from the beginning.
By Friday of our sprint, we had two competing solutions. Merci Grace, Slack’s product manager, favored a step-by-step guide called “The Tenacious Tour” that would show customers how Slack worked. Stewart Butterfield, Slack’s founder and CEO, had a hunch about an idea called “Bot Team”—a way for customers to try out Slack by communicating with a team of computer-controlled characters. You know this story, but you don’t know how it ends.
That’s what Friday is all about—finding the end to your sprint story. It’s your chance to put your prototypes in front of real customers, see how they react, answer your sprint questions, and make a plan for what to do next. On that Friday, everyone was excited and a bit nervous to see how the prototypes would do. A buzz of quiet chatter stopped when the screen flickered to life.
Over the video stream, we heard a door close. Michael’s voice began, “Thanks again for coming in today.” Then we saw the first customer sit down, nervously look right into the camera, and relax as Michael asked a few warm-up questions.
Michael introduced the customer to our first prototype. For a moment, the customer did nothing, and then she leaned forward, grabbed the computer mouse, and started talking.
• • •
Friday feels like one long mystery. Throughout the day, you’ll collect clues. Some of those clues help you crack the case, but some lead you in the wrong direction. It’s only at the end—around 5 p.m.—that everything ties together and the answers become clear.
Just like the Slack team, your sprint team will spend Friday together. While the Interviewer is testing the prototype with customers, your team will gather in the sprint room to watch and take notes. It’s the end of an intense week, and your team might feel pressure to get back to “normal work”—emails, meetings, and mission-critical water cooler conversation. But the sprint only works if you stick together until the end.
Everybody has a superpower. A unique strength. For software engineers, it’s writing code. For marketers, it might be designing campaigns. For us, it’s putting sticky notes on whiteboards. There’s one skill that you’re especially good at, and you probably feel most productive when you’re doing that one thing.
It’s tempting to disband the sprint group on Friday and let everyone return to his or her superpower. That way, the Interviewer can use his interviewing superpower to test your prototype with customers. We’ve tried this approach, and here’s how it goes: The Interviewer talks to each customer. Good so far. Unfortunately, he can’t talk and take detailed notes at the same time, so he records the conversations. The interviews are on Friday, so the soonest he can listen to those recordings is Monday. The interviews take all day, so it takes another full day to review the recordings and make sense of what he heard. Then he needs a few hours to put together a document or presentation with his findings. Now it’s Tuesday. (We even know some researchers who edit together a video “highlight reel” of the most interesting moments from the interviews—awesome, but time-consuming.) And once all that’s done, the Interviewer needs to schedule time with the sprint team to present and review the findings. At the earliest, it will be the Wednesday after the sprint before the team sees the results.
There are other problems. As time passes, the team’s momentum will disintegrate as everyone gets sucked into the vortex of business as usual. And there’s a credibility problem, too. Because the team didn’t witness the test, they’re stuck trusting the Interviewer’s process and results. It’s like the difference between watching a movie yourself and just having someone tell you about it.
Luckily, the solution to all these problems is simple: Watch the interviews together. It’s much faster, because everyone is absorbing the results at once. Your conclusions will be better as a group, since you have seven brains working together. You’ll avoid problems of credibility and trust, because each sprinter can see the results with his or her own eyes. And at the end of the day, your team can make an informed decision about what to do next—the results of the interviews (and the sprint) are still clear in everyone’s short-term memory.
This wonderful teamwork doesn’t happen by itself, but with a few simple steps, you can create it every time. Here’s what to do:
Before the first interview begins, draw a grid on a large whiteboard in the sprint room. Create five columns—one for each customer you’ll be interviewing—and a few rows—one for each prototype, or section of the prototype, or sprint question you’re trying to answer.
Distribute sticky notes and whiteboard markers to everyone in the room. Give everyone instructions for how to take notes during the interviews: “When you hear or see something interesting, write it down on a sticky note. You can write down quotes, observations, or your interpretation of what happened.”
Use a different color marker depending on the note: green for positive, red for negative, black for neutral. If you only have black markers, write a plus or minus in the corner, or leave it blank for neutral.
During the interviews, the room should be quiet. The interview itself is a time for careful listening and detailed note-taking, not boisterous reactions or problem solving on the spot. It’s also important to be respectful of the customer being interviewed. Even though the customer can’t hear you (the video should stream “one way”) keep in mind that if she struggles with your prototype, it’s your problem, not the customer’s.
At the end of each interview, collect the notes and stick them to the whiteboard. Put them into the correct row and column, but don’t worry about any other organizing just yet. Then, take a break. Focusing and taking notes for five hours is tiring, so get some downtime between each interview.
• • •
By Friday afternoon, five target customers had tried the two Slack prototypes, and the whiteboard was covered with sticky notes. We gathered around to organize them and look for patterns.
We started by looking at the reactions to “The Tenacious Tour,” the solution that featured a straightforward, step-by-step guide to Slack. There was still general confusion about how Slack worked with email, but four out of five customers had understood the overall value—a huge success. Just two of the five had tried to sign up, but there appeared to be many easy-to-fix problems that might improve that number. (One forehead slapper: the “Tour” sign-up button was too far down the page.) Everyone agreed: “The Tenacious Tour” wasn’t perfect, but it was way better than Slack’s current marketing.
Then we shifted our attention to the results for “Bot Team.” Customer by customer, we read the notes. It wasn’t pretty. The observations were dominated by comments like “She’s confused,” “Doesn’t seem better than email,” and “I’m not really sure what this is.” Only one person had enjoyed talking to the computer-controlled characters, and even that person was bewildered by the purpose of the software.
We’d all watched these interviews, of course, but as we looked at the notes, it sunk in: Stewart’s hunch had been wrong. That was a surprise—Stewart’s intuition was normally excellent—but it was also a relief. Building “Bot Team” and getting it right would have been a big, expensive undertaking. We had given the realistic prototype our best effort, and it had failed. Now the whole team felt confident about focusing elsewhere.
On the other hand, “The Tenacious Tour” looked promising. The pieces were there, and some of the problems would be easy to fix. The next step was obvious. To close the gap, Merci and her team would run another sprint.
• • •
The Slack team had hoped for a breakthrough success; instead they got mixed results. But there was good news: they knew “The Tenacious Tour” was an improvement, they knew “Bot Team” was trouble, and they knew they had to focus on the “Slack vs. email” question.
Turning a whiteboard full of sticky notes into a list of patterns and next steps may sound like alchemy, but when everyone has watched the interviews together, it’s straightforward.
Ask the entire team to gather near the whiteboard. Everyone should stand close enough to read the sticky notes. Take about five minutes to silently review the notes; give each person a notepad and pen to write down patterns he or she sees. Look for patterns that show up with three or more customers. If only two customers reacted in the same way but it was an especially strong reaction, make note of that, too.
After five minutes looking for patterns individually, ask the team to share what they found and read the patterns aloud. On another whiteboard, list every pattern and label each one as positive, negative, or neutral. Once the patterns are listed, it’s time to make sense of the results.
On Monday, you came up with a list of sprint questions. These are the unknowns that stand between your team and your long-term goal. Now that you’ve run your test and identified patterns in the results, it’s time to look back at those sprint questions. These questions will help you decide which patterns are most important, and also point you toward next steps.
Slack had two big sprint questions. First, they wondered, “Can we explain Slack to people who have never tried it?” After the sprint, the answer was “Yes . . . maybe.” “The Tenacious Tour” had done a decent job of explaining Slack. But Merci and the rest of the team weren’t satisfied with a “decent job.” They wanted to fix “The Tenacious Tour” and make it even better.
Their second question was “Can we help an individual understand Slack before their team joins?” Every team that adopts Slack starts with a single person. That person has to imagine what it will be like to use the software with a whole team before convincing his or her coworkers to join. The fake team in “Bot Team” was an attempt to solve this problem, and it was a failure. Still, Slack thought there might be a different way to approach this challenge on the marketing page. So they answered, “No . . . maybe,” and vowed to try again in their next sprint.
At the end of your own sprint, you’ll do the same. Review your long-term goal and sprint questions from Monday. You probably won’t answer every question, but like Slack, you’ll make progress.
After looking back, it’s usually easy to figure out the next step. The team can have a short discussion, and then (you guessed it) the Decider decides how to follow up.
Maybe the best part about a sprint is that you can’t lose. If you test your prototype with customers, you’ll win the best prize of all—the chance to learn, in just five days, whether you’re on the right track with your ideas. The results don’t follow a neat template. You can have efficient failures that are good news, flawed successes that need more work, and many other outcomes. Let’s look at how five teams interpreted their test results, and what they decided to do next.
Slack had two outcomes from their sprint. First, they had an efficient failure when they discovered that one solution didn’t work, saving months of engineering work and extraordinary cost. Their other prototype was a flawed success. Three weeks later, the team reconvened for a follow-up sprint to improve on “The Tenacious Tour.” They better explained how messaging worked. They improved their diagrams and clarified the guide. When they tested the improved prototype, the results were stark: The new website was understood by five out of five customers, and Slack built and launched it afterward.
The robot makers Savioke had a sprint with a rare outcome—virtually every idea we tested was successful. Afterward, the team poured their efforts into bringing those ideas to market, and it paid off in the form of great press coverage and new hotel customers.
Blue Bottle Coffee tested three competing prototypes in a classic Rumble. One idea was an efficient failure; the other two were flawed successes. Blue Bottle took the best elements from those two winners and merged them into a website that dramatically increased sales.
Flatiron’s sprint question was a big one: Would cancer clinics change their workflow to use a new tool? The stakes were high. If they convinced research coordinators to switch, they could enroll more patients in clinical trials. Working together, we prototyped new software, then tested it with research coordinators. The result was an exciting flawed success. The coordinators didn’t love every part of the prototype, but their enthusiastic reaction to the concept gave Flatiron the confidence to continue designing and developing the software. Six months later, clinics were using the real thing to match patients to trials.
Many times, a successful test is not the end of the process, but the beginning. In 2014 we ran a sprint with Medium, a writing platform created by Twitter founder Ev Williams. Ev and his team had several ideas for improving Medium’s commenting and discussion tools, and after Friday’s test, there were several flawed successes worth pursuing. Medium’s engineering team spent the next week building two of the strongest ideas from the sprint. Then, as a test, they launched them to a fraction of Medium’s users. It was a follow-up sprint with large-scale data. (It turned out that both ideas increased discussion.)
Many companies want to launch quickly so they can get data from hundreds, thousands, or even millions of people. That large-scale data is great. But in the rush to get there, it’s easy to miss the opportunity to gather small-scale data early, when there’s still time to course-correct. As Medium’s story illustrates, you can have the best of both worlds. You can talk to your customers, and you can learn from large-scale data.
When you get into a regular rhythm of listening to customers, it can remind you why you’re working so hard in the first place. Every interview draws you and your team closer to the people you’re trying to help with your product or service.
If you continue running sprints, and if you’re true to your vision, the day will come when you’ll close that gap. You’ll be watching some Friday’s test, and you’ll see people understand your idea, believe it will improve their lives, and ask the Interviewer how to buy it.
In these moments, it’s like Mission Control cheering when the Apollo 13 module safely splashes down in the Pacific. It’s like the thieves from Ocean’s Eleven watching the fountain after the heist, or Gandalf swooping in on a giant eagle to rescue Frodo and Sam. It’s amazing. It’s what work should be about—not wasting time in endless meetings, then seeking camaraderie in a team-building event at a bowling alley—but working together to build something that matters to real people. This is the best use of your time. This is a sprint.