Every experiment has strengths and weaknesses. Some are quick and cheap but produce less reliable evidence. Some produce more reliable evidence but require more time and money to execute.
Consider cost, data reliability, and time required when you design your mix of experiments. As a rule of thumb, start cheap when uncertainty is high and increase your spending on experiments with increasing certainty.
Select a series of tests by drawing from our experiment library or by using your imagination to invent new experiments. Keep two things in mind when you compose your mix:
What customers say and do are two different things.
Use experiments that provide verbal evidence from customers as a starting point. Get customers to perform actions and engage them (e.g., interact with a prototype) to produce stronger evidence based on what they do, not what they say.
Customers behave differently when you are there or when you are not.
During direct personal contact with customers, you can learn why they do or say something and get their input on how to improve your value proposition. However, your presence might lead them to behave differently than if you weren’t there.
In an indirect observation of customers (on the web, for example) you are closer to a real-life situation that isn’t biased by your interaction with customers. You can collect numerical data and track how many customers performed an action you induced.
Produce Evidence with a Call to Action
Use experiments to test if customers are interested, what preferences they have, and if they are willing to pay for what you have to offer. Get them to perform a call to action (CTA) as much as possible in order to engage them and produce evidence of what works and what doesn’t.
The more a customer (test subject) has to invest to perform a CTA, the stronger the evidence that he or she is really interested. Clicking a button, answering a survey, providing a personal e-mail, or making a prepurchase are different levels of investments. Select your experiments accordingly.
CTAs with a low level of investment are appropriate at the beginning of value proposition design. Those requiring a high level of investment make more sense later in the process.
Ad Tracking
Use ad tracking to explore your potential customers’ jobs, pains, gains, and interest—or lack of it—for a new value proposition. Ad tracking is an established technique used by advertisers to measure the effectiveness of ad spending. You can use the same technique to explore customer interest even before a value proposition exists.
Test customer interest with Google AdWords
We use Google AdWords to illustrate this technique because it’s particularly well suited for testing based on its use of search terms for advertising (other services such as LinkedIn and Facebook also work well).
Select search terms. Select search terms that best represent what you want to test (e.g., the existence of a customer job, pain, or gain or the interest for a value proposition).
Design your ad/test. Design your test ad with a headline, link to a landing page, and blurb. Make sure it represents what you want to test.
Launch your campaign. Define a budget for your ad/testing campaign and launch it. Pay only for clicks on your ad, which represent interest.
Measure clicks. Learn how many people click on your ad. No clicks may indicate a lack of interest.
Unique Link Tracking
Set up unique link tracking to verify potential customers’ or partners’ interest beyond what they might tell you in a meeting, interview, or call. It’s an extremely simple way to measure genuine interest.
Where to apply?
MVP Catalog
MVP stands for minimum viable product, a concept popularized by the lean start-up movement to efficiently test the interest in a product before building it entirely. Rather than coining a new term we stick to this established one and adapt it to testing value propositions.
Use the following techniques to make your value propositions feel real and tangible before implementing anything when you test them with potential customers and partners.
Use prototypes designed specifically to learn from experiments with potential customers and partners.
Illustrations, Storyboards, and Scenarios
Share illustrations, storyboards, and scenarios related to your value proposition ideas with your potential customers to learn what really matters to them. These types of Illustrations are quick and cheap to produce and make even the most complex value propositions tangible.
Life-Size Experiments
Get your customers to interact with life-size prototypes and real-world replicas of service experiences. Stick to the principles of rapid, quick, and low-cost prototyping to gather customer insights despite the more sophisticated set-up. Add a CTA to validate interest.
Concept Cars and Life-Size Prototypes
These are cars made to showcase new designs and technologies. Their purpose is get reactions from customers rather than to go into mass production directly.
Lit Motors used lean start-up principles to prototype and test a fully electric, gyroscopically stabilized two-wheel drive with customers. Because this type of vehicle represents a completely new concept, it was essential for Lit Motors to understand customer perception and acceptance from the very beginning.
In addition, Lit Motors added a CTA to validate customer interest beyond initial interactions with the prototype. Customers can prereserve a vehicle with a deposit ranging from $250 to $10,000. Deposits go into a holding account until vehicles are ready, with higher deposits moving customers to the front of the waiting list.
Prototype Spaces
These are spaces to cocreate products and service experiences with customers and/or observe their behavior to gain new insights. Invite potential customers to create their own perfect experience. Include industry experts to help build and test new concepts and ideas.
Hotel chain Marriott built a prototyping space in its headquarters’ basement called the Underground. Guests and experts are invited to create the hotel experience of the future by cocreating hotel rooms and other spaces. Guests are invited to add furniture, electricity outlets, electronic gadgets, and more, to hotel room replicas that can easily be reconfigured.
Landing Page
The typical landing page MVP is a single web page or simple website that describes a value proposition or some aspects of it. The website visitor is invited to perform a CTA that allows the tester to validate one or more hypotheses. The main learning instrument of a landing page MVP is the conversion rate from the number of people visiting the site to visitors performing the CTA (e.g., e-mail sign-up, simulated purchase).
When?
Test early to learn if the jobs, pains, and gains you intend to address and/or your value proposition are sufficiently important to your customer for them to perform an action.
Variations
Combine with split testing to investigate preferences or alternatives that work better than others. Measure click activity with so-called heat maps to learn where visitors click on your page.
Design your landing page, traffic generation, and CTA based on your learning goals.
Split Testing
Split testing, also known as A/B testing, is a technique to compare the performance of two or more options. In this book we apply the technique to compare the performance of alternative value propositions with customers or to learn more about jobs, pains, and gains.
Innovation Games®
Innovation Games is a methodology popularized by Luke Hohmann to help you design better value propositions by using collaborative play with your (potential) customers. The games can be played online or in person. We present three of them.
All three Innovation Games we present can be used in various ways. We outline three specific tasks they can help us with when it comes to the Value Proposition Canvas and related hypotheses.
Hohmann, Innovation Games, 2006.
Speed Boat
This is a simple but powerful game to help you verify your understanding of customer pains. Get your customers to explicitly state the problems, obstacles, and risks that are holding them back from successfully performing their jobs to be done by using the analogy of a speed boat held back by anchors.
Product Box
In this game, you ask customers to design a product box that represents the value proposition they’d want to buy from you. You’ll learn what matters to customers and which features they get excited about.
Buy a Feature
This is a sophisticated game to get customers to prioritize among a list of predefined (but not yet existing) value proposition features. Customers get a limited budget of play money to buy their preferred features, which you price based on real-world factors.
Mock Sales
A great way to test sincere customer interest is to set up a mock sale before your value proposition even exists. The goal is make your customers believe they are completing a real purchase. This is easily done in an online context but can also be done in a physical one.
Presales
The main objective of this type of presales is to explore customer interest; it is not to sell. Customers make a purchase commitment and are aware of the fact that your value proposition does not yet fully exist. In case of a lack of interest, the sale is canceled and the customer reimbursed.
