25

Execution Interview

The execution interview evaluates your ability to set goals, make effective trade-offs, and identify root causes.

What Your Interviewer Is Looking For

During the execution interview, your interviewer is looking for:

1. Goal setting: Can you define a goal metric for the product that aligns with customer and company goals? Can you evaluate whether your goal is a good one?
2. Trade-offs: Can you align on goals with another team? How do you make decisions and navigate trade-offs?
3. Root cause: Can you take a broad problem and break it down into smaller pieces to identify the root cause?

Execution Frameworks

Just like the product sense interview, you should answer execution questions in a structured way. Here are several frameworks that you can use :

Setting Goals

Ask clarifying questions and explain your approach.

1. Problem

2. Company

3. Solution

a) Map out the user journey

b) Brainstorm metrics

c) Select a goal metric

4. Evaluate the goal

a) Customer and company goals?

b) Short-term or long-term goal?

c) Can it be gamed?

d) Can you team drive it?

Summarize.

Evaluating Trade-Offs

There are many different ways to answer trade-off questions. Here are a few best practices based on what we learned in previous chapters:

1. Start with why. Try to align on shared goals first.
2. Gather more information. Understand the pros and cons of each trade-off (e.g. can you run a simple experiment as a path forward?)
3. Disagree and commit or escalate. If two people or teams don't have shared goals, it becomes tough to agree on a path forward. In these situations, you need to either disagree and commit or escalate to your management chain .

Root Cause Analysis

If a metric declined, evaluate:

1. Logging issue

2. Time period

3. Platform

4. Location

5. Externalities

6. Customer journey

Summarize.

Let’s walk through an example interview that uses all three frameworks above.

Goal Setting Example

You’re the product manager for Facebook Groups. What goal will you set?

Ask clarifying questions.

Candidate : Thank you! By groups, are you referring to public groups in the main Facebook app or private groups that I can form with friends on Messenger?

Interviewer : Good question. I’m referring to the groups product in the main Facebook app, but these groups can be both public or private.

Explain your approach .

Candidate : Thanks! Ok, here’s how I plan to tackle this problem. First, I want to better understand our customers and their pain points. Second, I want to know why Groups matters to Facebook. Third, I’d like to walk through the user journey and brainstorm a possible goal metric. Finally, I’d like to evaluate whether the metric makes sense. Does that sound good?

Interviewer : Yes, let’s proceed.

Whiteboard :

1. Problem

2. Company

3. Solution

4. Goal

1. Problem

Candidate : Great, so I think there are two types of customers for Facebook groups: creators who make groups and members who join and participate in groups. There are also many different types of groups. For example, I’m part of a new dad group where I get a lot of baby advice and a shopping group for a brand that I like.

At a high level, I think people create and join groups to be part of a community that shares a common interest or identity. Do you agree?

Interviewer : Yes, that makes sense.

Whiteboard :

1. Problem

Customers: Creators, members

Types of groups: Interest, commerce, local, friend

Summary: Be part of a community that shares a common interest or identity .

2. Company

Candidate : Ok, now let’s discuss why Groups matters to Facebook. Facebook’s mission is to bring the world closer together. Groups is a key part of this mission because it brings people together around a common interest or identity—whether that’s a group for new parents, shoppers, or people from a local neighborhood. At a time when social media is getting attention for dividing people, I think Groups can help people create and be part of communities that matter to them.

In terms of metrics, can you tell me what overall metrics the company tracks?

Interviewer : What do you think?

Candidate : In the past, I would have said metrics like time spent. But I think what Facebook cares about now is helping people have meaningful interactions. This metric directly relates to Facebook’s mission to bring the world closer together.

Interviewer : Yes, well, we still care about time spent, but we want people to spend time on Facebook forming meaningful connections and interactions.

Candidate : Ok sounds good. Let me write this down.

Whiteboard :

2. Company

FB: Bring the world closer together.

Metric: Meaningful interactions

3. Solution

Candidate : Great, now before we pick a metric to set a goal for, I just want to map out the full user journey for groups.

Whiteboard :

Journey / metrics

1. Create groups

2. Discover / join groups

3. Read group posts

4. Interact in groups

Candidate : Now I’d like to brainstorm metrics for each step of the journey.

Whiteboard :

1. Create groups

a) Daily active group creators

b) # of groups created

2. Discover / join groups

a) # group joins

3. Read group posts

a) # people who create group content

b) # people who read group posts

c) Time spent reading group posts

4. Interact in groups

a) CTR for link and photo group posts

b) Likes / comments / shares on group posts

c) Hide story actions for groups

4. Evaluate the goal.

Candidate : As you can see, there are a lot of metrics here. Let’s use the following criteria to evaluate which goal metric we should choose.

Whiteboard :

4. Evaluate metric

a) Customer / company goals?

b) Short-term or long-term goal?

c) Can metric be gamed?

d) Can the team drive it?

Candidate : First, we should choose a metric that aligns with customer and company goals. Earlier, we mentioned that both group creators and members want to be part of a community that shares a common interest or identity. We also know that Facebook’s mission is to bring the world closer together, with meaningful interactions as a goal metric. Given all this, I would want to look at the “interact in groups” step of the user journey. Of the metrics that we brainstormed for this step, I think measuring comments on group posts is interesting. Comments usually take more thought than likes and shares and are more likely to be a meaningful interaction. Instead of just looking at total comments, I think setting a goal for daily active group commenters makes sense. We want as many people to have meaningful interactions as possible.

Interviewer : That’s interesting. I understand why you think comments might be meaningful, but there are also a lot of comments on Facebook and other platforms that aren’t meaningful.

Candidate : That’s a good point. Facebook does require users to give their real identity, so comments in Facebook groups should be more meaningful than comments on other platforms. That said, I think we should track the number of comment likes, hides, and reports as secondary metrics to monitor comment quality.

Interviewer : Ok let’s keep going .

Candidate : Next, let’s look at whether this is a short-term or long-term goal. Given that there are already a lot of groups active on Facebook, I think selecting daily group commenters as a short-term goal is OK. If, on the other hand, there aren’t a lot of groups, then I would probably pick a goal earlier in the user journey like daily groups created.

Interviewer : That’s fair.

Candidate : Now, let’s think about how we can game daily group commenters. We already discussed that not all comments are good and that we should monitor negative signals like comment hides or user reports. Just from using Reddit, I know that having moderators is critical to ensuring that discussions remain productive and on topic. I would want to measure daily active moderators as another metric.

Interviewer : You’re measuring a lot of metrics. I want to make sure that the daily group commenters metric is still your goal metric?

Candidate : Yes, I think so. Comment hides, user reports, and daily moderators are useful health metrics to monitor, but our goal metric should still be daily commenters.

Summarize.

Interviewer : Ok sounds good. Do you want to wrap this up?

Candidate : Yes. The mission for Facebook Groups is to help people create and be part of communities that matter to them. This mission relates directly to Facebook’s mission to bring the world closer together. After mapping out the user journey for groups, we decided to use daily group commenters as our goal metric. This goal ties directly to Facebook’s mission and helps grow meaningful interactions. However, we want to make sure that we’re also monitoring the quality of comments. We will track comment likes, hides, reports, and daily moderators as secondary metrics.

Trade-Off Example

Ok, now imagine that you’re trying to grow daily active commenters for groups by surfacing group posts in News Feed. When trying to do this, you come into conflict with the video team who wants to grow video engagement on Facebook. What do you do?

Candidate : Hmm, that’s a good question. How does the video team measure engagement?

Interviewer : They measure engagement using video watch time and the number of comments on video posts.

Candidate : Ok. How about the News Feed team?

Interviewer : The News Feed team cares a lot about meaningful interactions since News Feed is the most critical surface on Facebook. To them, a meaningful interaction is either a comment or a reaction (like).

Candidate : Thank you! So, it sounds like the trade-off is this: if we promote group posts in News Feed, then video posts could get less visibility. There isn’t a way to encourage both video and group posts?

Interviewer : Let’s assume that there isn’t.

Candidate : Ok, first, I would try to align all three teams on the same goal metric. It sounds they all care about growing comments as a goal.

Interviewer : Yes that’s true, but the video team also cares about watch time and the News Feed team also cares about reactions.

Candidate : I think we should ask the video and News Feed teams to let us run an experiment to promote group posts in Feed. For this experiment, we'll set growing the number of daily active commenters as the primary goal while also monitoring video watch time and reactions. If daily active commenters grow significantly while the other two metrics are stable, I think we should be able to ship the experiment.

Interviewer : Let’s say comments grow significantly, but video watch time declines. As a result, the video team is not happy.

Candidate : In that case, I would explore two paths:

1. Run another experiment promoting group video posts to see if it improves both watch time and comments.

2. Explore other channels (e.g. notifications) to grow group posts without coming into conflict with the video team.

Interviewer : Ok, let’s say these two options don’t work out and the video team is still blocking your launch. Now what?

Candidate : At this point, I would work with the video team to escalate this decision to someone who is a leader for both teams. The problem here is that both sides have different goals. I would rather escalate quickly and get an answer instead of arguing back and forth between two teams with misaligned goals.

Root Cause Example

Great. Now imagine that daily active commenters have declined 10% over the past month. How would you understand why?

Candidate : Thanks. I would work with my analyst to investigate the following:

Whiteboard :

1. Logging issu e

2. Time period

3. Platform

4. Location

5. Externalities

6. Customer journey

Candidate : My first question would be, is this a logging issue?

Interviewer : Let’s assume that logging is correct.

Candidate : Got it. Now let’s look at the time period. You mentioned that commenters declined over 30% in the most recent month. Could this be due to seasonality? I would look at the same month last year as a comparison.

Interviewer : There is some seasonality but it doesn’t really explain the 10% drop.

Candidate : Interesting. When during the month did the 10% drop happen? Was it a steady drop throughout the month or did it drop drastically on one particular day?

Interviewer : After some investigation, we found that the majority of the drop came from a weekend in the middle of the month.

Candidate : Hmm, I wonder what happened that weekend? Did the drop happen across all our platforms?

Interviewer : No, the majority of the drop came from Android.

Candidate : Did something break on our Android app?

Interviewer : What do you mean?

Candidate : For example, did the Facebook app fail to load that weekend or something broke with the user journey for groups? Create groups, join groups, comment in groups, etc .

Interviewer : No, we didn’t see any significant product errors during that time.

Candidate : Ok, did the drop happen in a particular region or country?

Interviewer : Yes, commenters in Brazil declined drastically.

Candidate : Very interesting. What happened in Brazil that weekend?

Interviewer : There was an election that weekend and two of our largest groups in Brazil were shut down for violating our community guidelines.

Candidate : Wow! Ok, I would look into why those groups were shut down but if that’s true, then it makes sense that our commenters metric would drop.

Being able to execute is a fundamental requirement for all product managers. I hope that this example interview gives you an idea of how to answer execution questions well.