Michael Margolis is an excellent conversationalist. He smiles easily and asks lots of questions, brimming with a natural curiosity about what it’s like to live where you live, work where you work, and do whatever it is that you do. It’s only afterward that you realize you were talking the whole time and learned little about him.
Michael’s friendliness and curiosity are genuine, but his conversational skills aren’t just a natural gift. Michael is a research partner at GV, and when you watch him interviewing customers—which we’ve seen him do hundreds of times—you realize it’s a practiced art. Everything from the structure of his questions to his body language helps people think aloud and express themselves honestly.
For more than twenty-five years, Michael has conducted research for all kinds of companies—Electronic Arts, Alcoa, Sun Microsystems, Maytag, Unilever, Walmart.com, and Google. Since 2010, he’s been at GV, working with the startups in our portfolio.
Over the years, Michael has adapted his research methods to be fast enough for startups, and learnable for the people who work there. Michael has trained product managers, engineers, designers, salespeople, and countless others in how to conduct these interviews. Anyone can do it—even a CEO.
In this chapter, we’ll let you in on some of Michael’s secrets. Back on Tuesday, you learned his shortcuts for recruiting the perfect target customers (see pages 119–123). In this chapter, you’ll learn how to interview. These interviews can teach you about the people who use your product, reveal hidden problems with your solutions, and uncover the “why” behind it all.
No matter what kind of customer he’s talking to, or what kind of prototype he’s testing, Michael uses the same basic structure: the Five-Act Interview.
This structured conversation helps the customer get comfortable, establishes some background, and ensures that the entire prototype is reviewed. Here’s how it goes:
1. A friendly welcome to start the interview
2. A series of general, open-ended context questions about the customer
3. Introduction to the prototype(s)
4. Detailed tasks to get the customer reacting to the prototype
5. A quick debrief to capture the customer’s overarching thoughts and impressions
Friday’s action takes place in two rooms. In the sprint room, the team watches the interviews over live video. (Nothing sneaky here. You’ll get the customer’s permission to record and play the video.) The interview itself takes place in another, smaller room—which we cleverly call the “interview room.”
There’s no special tech setup required. We use a regular laptop with a webcam and simple video meeting software to share the video and audio. This arrangement works for websites, but it also works for mobile devices, robots, and other hardware—just point the webcams at what you want to see.
Michael Margolis conducting an interview. He sits beside the customer, but gives her plenty of space. A webcam streams video of the customer’s reaction to the sprint room.
As complicated as it gets: When testing mobile apps or hardware devices, we use a document camera connected to a laptop. Video streams from the laptop to the sprint room.
Sometimes the Interviewer or the customer are in another building, another city, or out in the field (Michael has conducted interviews at hospitals, hotels, and truck stops), but since the sprint team is watching over video, that doesn’t matter. What does matter is that the Interviewer and the customer are sitting side by side, talking comfortably. The interview is a not a group exercise; it’s a conversation between two people. One person from your team can be the Interviewer for the entire day, or two people can alternate. (Since you’re looking for big, obvious patterns, you don’t have to worry about tainting the data with this kind of small change.)
People need to feel comfortable to be open, honest, and critical. So the first job of the Interviewer is to welcome the customer and put her at ease. That means a warm greeting and friendly small talk about the weather. It also means smiling a lot. (If you’re not in the mood to smile, prepare for the interview by listening to “Keep A-Knockin’ ” by Little Richard.)
Once the customer is comfortably seated in the interview room, the Interviewer should say something like:
“Thanks for coming in today! We’re always trying to improve our product, and getting your honest feedback is a really important part of that.
“This interview will be pretty informal. I’ll ask a lot of questions, but I’m not testing you—I’m actually testing this product. If you get stuck or confused, it’s not your fault. In fact, it helps us find problems we need to fix.
“I’ll start by asking some background questions, then I’ll show you some things we’re working on. Do you have any questions before we begin?”
The Interviewer should also ask the customer if it’s okay to record and watch the video of the interview, and he or she should make sure the customer signs any legal paperwork insisted on by your lawyers. (We use a simple one-page form for nondisclosure, permission to record, and invention assignment. These forms can also be signed electronically before the interview.)
After the introduction, you’ll be eager to bring out the prototype. Not so fast. Instead, start slow by asking some questions about the customer’s life, interests, and activities. These questions help build rapport, but they also give you context for understanding and interpreting your customer’s reactions and responses.
A great series of context questions starts with small talk and transitions into personal questions relevant to the sprint. If you do it right, customers won’t realize the interview has started. It will feel just like natural conversation.
In our sprint with FitStar, we knew it would be helpful to understand more about each customer’s approach to exercise. Michael’s context questions went something like this:
“What kind of work do you do?”
“For how long have you been doing that?”
“What do you do when you’re not working?”
“What do you do to take care of yourself? To stay in shape? To stay active?”
“Have you used any apps or websites or other things to help with fitness? Which ones?”
“What did you want them to do for you? What do you like or dislike about them? Did you pay for them? Why? Why not?”
As you can see, Michael started with generic small talk (“What kind of work do you do?”) then steered the topic to fitness (“What do you do to take care of yourself?”). As he asked each open-ended question, he encouraged answers with smiles, nods, and eye contact.
At minimum, these context questions make the customer more comfortable and forthcoming. But quite often, the answers help you understand how your product or service fits into the customer’s life—and perhaps, what people think about your competition. In the FitStar interviews, we learned about customers’ experience with workout videos and personal trainers, and how they exercised when they traveled—all useful information.
Now you’re ready to get the customer started on the prototype. Michael begins by saying:
“Would you be willing to look at some prototypes?”
By asking for permission, he reinforces the status relationship: The customer is doing him a favor, not the other way around, and it is the prototype that will be tested, not the customer. It’s also important to say:
“Some things may not work quite right yet—if you run into something that’s not working, I’ll let you know.”
Of course, if you built a “Goldilocks quality” prototype on Thursday, the customer will forget it isn’t real once they start using it. However, introducing it this way encourages them to give blunt feedback. Explaining that it’s a prototype also makes the Interviewer’s job easier in case something breaks or the customer encounters a dead end (both of which are likely to happen).
Remind the customer that you’re testing the prototype—not her:
“There are no right or wrong answers. Since I didn’t design this, you won’t hurt my feelings or flatter me. In fact, frank, candid feedback is the most helpful.”
That “I didn’t design this” line is important, because it’s easier for customers to be honest if they don’t think the Interviewer is emotionally invested in the ideas. Hopefully the Interviewer avoided working on the prototype on Thursday—but he should probably say “I didn’t design this” even if he actually did. Don’t worry, we won’t tell on you.
The Interviewer should also remind the customer to think aloud:
“As we go, please think aloud. Tell me what you’re trying to do and how you think you can do it. If you get confused or don’t understand something, please tell me. If you see things you like, tell me that, too.”
Thinking aloud makes the interview format especially powerful. Seeing where customers struggle and where they succeed with your prototype is useful—but hearing their thoughts as they go is invaluable.
In the real world, your product will stand alone—people will find it, evaluate it, and use it without you there to guide them. Asking target customers to do realistic tasks during an interview is the best way to simulate that real-world experience.
Good task instructions are like clues for a treasure hunt—it’s no fun (and not useful) if you’re told where to go and what to do. You want to watch customers figure out the prototype on their own. As an example, here’s the task from the FitStar test:
“Let’s say you came across FitStar in the App Store. How would you decide if you wanted to try it?”
Starting from this simple nudge, the customer reads and evaluates the app description, installs the app, and tries it out. The “how would you decide?” phrasing encourages her to act naturally along the way.
We learned much more from this simple task than we would have if Michael had micromanaged her at every step. (“Install the app. Now sign up. Now fill in your name.”) Open-ended tasks lead to interesting interviews. Overly specific tasks are boring for both the customer and the sprint team.
As the customer goes through the task, the Interviewer should ask questions to help her think aloud:
“What is this? What is it for?”
“What do you think of that?”
“What do you expect that will do?”
“So, what goes through your mind as you look at this?”
“What are you looking for?”
“What would you do next? Why?”
These questions should be easy to answer and not intimidating. The Interviewer tries to keep the customer moving and thinking aloud, not anxious to find the right answer.
To wrap up the interview, ask a few debrief questions. You’ll see and hear a lot during each interview, and it can be tough to pick out the most important reactions, successes, and failures. When you ask debrief questions, your customers can help you sift through everything you heard.
Here are some of Michael’s debrief questions:
“How does this product compare to what you do now?”
“What did you like about this product? What did you dislike?”
“How would you describe this product to a friend?”
“If you had three magic wishes to improve this product, what would they be?”
Don’t worry—asking the “magic wishes” question doesn’t mean you’re turning your product planning over to your customers. Instead, it helps customers articulate their reactions. It will still be up to you to decide how to interpret and apply what you learn.
If you’re testing two or more prototypes in your interviews, review each one (to refresh the customer’s memory) and ask these questions:
“How would you compare those different products? What are the pros and cons?”
“Which parts of each would you combine to create a new, better version?”
“Which one worked better for you? Why?”
And that’s it. When the interview is over, the Interviewer thanks the customer, gives her a gift card, and shows her out.
Throughout the session, the Interviewer should remain engaged in the conversation. He should encourage the customer to talk while remaining neutral (say things like “uh-huh” and “mmm hmm,” not “great!” and “good job!”) There’s no need to take notes. The rest of the team in the sprint room will take care of that for you.
Of course, we don’t expect anyone to memorize every question and all five acts. On Thursday, the Interviewer can start early by writing up a script while the rest of the team is prototyping. On Friday, he can make a printout to refer to as the interview goes along. Not only will the script make the interviews easier to run, it will also make them consistent—which makes it easier to spot patterns throughout the day.
• • •
One of our favorite stories about the power of interviews comes from our friend, a designer named Joe Gebbia. Back in 2008, Joe and a couple of friends founded a startup. They had what they thought was an amazing idea for a new online marketplace. They built and launched a website, then spent months improving it until they were pretty sure it was perfect.
But despite their efforts, the new service wasn’t catching on. They had a few customers and a little revenue, but they weren’t growing, and—bringing in just $200 a week—they weren’t even making enough to pay the rent. Hoping they could turn the business around before running out of money, the founders took a somewhat desperate measure. They stopped their engineering work, left the office, and tracked down a handful of their customers. Then they interviewed them. One at a time, face-to-face, they watched people use their website.
Joe describes those interviews as “agonizing and enlightening.” He recalls, “We were, like, smacking our heads.” Their website was riddled with flaws. Even simple issues—such as picking a date on a calendar—confused people.
When they returned to the office, Joe and his cofounders spent a week fixing the most glaring problems, and then released a new version to their customers. Revenue doubled to $400 a week, and Joe checked to make sure it wasn’t a bug in their accounting system. But the numbers were real. So they did another round of interviews, and another round of improvements. Revenue doubled again to $800, then $1,600, then $3,200 a week. That growth didn’t stop.
That startup was Airbnb. Today, the online hospitality marketplace operates in more than 30,000 cities and 190 countries. They’ve served more than 35 million guests. It turns out it was an amazing idea, but to make it work, they had to do those interviews. “There’s this gap between the vision and the customer,” Joe says. “To make the two fit, you have to talk to people.”
Airbnb’s interviews showed the founders how the product looked through their customers’ eyes, revealing problems the founders themselves couldn’t see. Listening to customers didn’t mean abandoning their vision. Instead, it gave them the knowledge they needed to combine with that vision, so they could close the gap and make a product that worked for real people.
We can’t promise that your interviews will make you as successful as Airbnb, but we can promise that the process will be enlightening. In the next chapter, we’ll talk about how to make sense of what you observe: taking notes, finding patterns, and drawing conclusions about next steps.
