Chapter Six
Start Small, Iterate Fast
Validation works best when you start small. The faster you can iterate, the faster your pace of improvement and innovation. By learning important lessons early, you can save time and money by avoiding investments in unnecessary infrastructure, manufacturing, and deployment. You can also minimize any potential risk from unintended consequences, particularly if you are working with vulnerable populations.
Programs in the social sector are often planned in painstaking detail, then deployed through large rollouts. This places a big bet on getting everything right off the bat. Inevitably, we don’t. Lean Impact replaces this linear process with an iterative one based on the scientific method. Design and implementation are melded into staged tests, each one building on the lessons of the last. Your solution should evolve, not from debates in a conference room, but from the data collected from the reactions and behaviors of real customers and stakeholders. Failure is a natural and essential part of the process.
This chapter will make the case for the Lean Impact principle of starting small, walk through the process of validation, and explore ways to accelerate learning. Once you confirm that you can deliver on all three pillars of social innovation – tangible customer value, an engine to accelerate growth, and meaningful social impact – you have a solid foundation to do more. As the saying goes, “Nail it before you scale it.”
LEARNING FROM FAILURE
Over the past 30 years the world has made dramatic progress, reducing the number of people living in poverty by more than half. Most of those gains were made in Asia, leaving extreme poverty increasingly concentrated in Africa, which is now home to half of the world’s poor.1 The majority live in rural areas and depend on agriculture for their livelihoods.
Smallholder farmers, who grow subsistence crops on small plots of land using family labor, are among the poorest and most neglected people on the planet. Despite numerous well‐meaning aid programs, the majority still live on less than two dollars a day and do not have access to the modern farming tools and techniques that have the potential to dramatically increase their crop yields and incomes. In 2006, after completing their MBAs at Northwestern University’s Kellogg School of Management, Andrew Youn and Matt Forti started the One Acre Fund with an aim to bridge this gap.
The One Acre Fund offers smallholder farmers in Africa a complete bundle of agricultural services, including delivery of improved seeds and fertilizer, training on good farming practices, and access to markets and crop storage facilities to boost their profits. By taking advantage of financing, farmers pay small amounts over time, covering most of the costs. As of 2017, One Acre’s core program was serving over 600,000 farmers, increasing their incomes on supported activities by an average of over 50%. It aims to bring these benefits to over a million farmers by 2020.
But it’s not resting on its laurels. The One Acre Fund has constantly sought ways to improve its offerings, and thereby farmers’ lives. Early on, in 2008, the team learned of an international buyer for passion fruit as an ingredient for drinks. Research and financial modeling indicated strong economics and potential profits. So they offered passion fruit as a new crop option to their farmer network.
Alas, the realities in the field didn’t match the idealized conditions of the researchers’ theoretical assumptions. Farmers were wary of this unfamiliar crop and needed extensive training to cultivate it properly. The result was poor quality fruit that fetched far lower prices than anticipated. Transport to ports from inland farms was also more expensive than projected. All in all, it was a failure.
The failure not only impacted the One Acre Fund as an organization, but also a large number of farmers who were already quite vulnerable. Andrew vowed to not let this happen again. He had learned that desk research wasn’t reliable and placing such a big bet on an unproven intervention was too risky. By staging risk more gradually, starting with small experiments, he could identify issues earlier and adapt as needed. This led him to create a new innovation framework.
Today, the One Acre Fund sources innovations from across its organization and network. Each idea is evaluated based on potential impact, the percentage of farmers likely to adopt it, the simplicity of the model, and operational feasibility. The best ideas are prototyped with a small number of farmers to learn from their real world experiences. This allows One Acre to iterate at a small scale and apply any necessary changes quickly, cheaply, and with less disruption. What fails, fails small. After prototyping, One Acre validates each of its four criteria through gradually larger trials and more realistic conditions – first a nursery setting, then a few dozen farmers’ fields, and finally at the scale of a district with thousands of farmers. Many innovations that initially appear promising are discontinued based on the results. By the time a new product is rolled out to the full farmer network, most of the potential risks are understood.
This process for research and development (R&D) has paid off. Based on the impacts achieved to date, the One Acre Fund estimates that it has generated a four‐ to six‐fold return on investment. One Acre is highly unusual in the nonprofit sector for its commitment to innovation, dedicating 7% of its annual budget to R&D. In the private sector, investors typically consider a company’s R&D spending to be a sign of health and a leading indicator of potential future growth. We should value R&D investments in social innovation and their potential to unlock far greater future impact similarly.
PRESSURE TO GROW
Common sense tells us that it’s better to fail small than to fail big. When we start by testing a solution with just a few customers, we can reduce risk, refine our idea, and abandon it if necessary without putting a lot of time, money, or reputation on the line. Then we can deliver a much better, validated solution to more and more people over time. So why do so many social interventions start to scale before they’re fully tested?
While all of us can get caught up in the thrill of a new innovation and be tempted to get ahead of ourselves, mission‐driven organizations face some unique pressures. For one, they’re typically either working with people who are suffering today or trying to avert a potential calamity tomorrow. Thus, the need to do something, anything, feels quite urgent. We get into this line of work because we want to make a difference, not stand on the sidelines. Our hearts and our spirits call us to action.
So when there is an opportunity to improve a situation, even just a bit, it is a natural human inclination to want to do as much as possible for as many as possible. Never mind that the consequences are unclear, the work is unsustainable, or a more cost‐effective alternative may exist. As a result, organizations tend to land on a solution that is “good enough” to help with the most immediate needs, then stick with it. Yet, by slowing down, starting small, and validating first, we could create far more benefit for far more people over time.
During my time as chief innovation officer at Mercy Corps, a global humanitarian aid nonprofit, I experienced this phenomenon firsthand. Our Social Ventures team raised an innovation fund in 2014 to invest in building our own social enterprises. Think of these as startups for social good. The goal was to create financially sustainable, mission‐aligned businesses that would continue to grow and thrive long after the initial funding was spent. Among the first was KibaruaNow, an online marketplace similar to TaskRabbit, that connected disadvantaged youth to short‐term work in Nairobi, Kenya. The youth‐population bulge and resulting high levels of unemployment is one of the biggest social challenges in Africa, so this business was well aligned with our mission.
It started out well. We found an amazing Kenyan leader with a passion for the business, experience tackling youth unemployment, and a postgraduate degree from a top African university to boot. She quickly recruited a local team of youth who were eagerly seeking work and clients who were willing to pay. The team created a simple vetting process to screen so‐called taskers and was off to the races. After testing a range of possible tasks, they learned that without a trusted relationship, customers were uncomfortable fronting the necessary cash for errands or entrusting a stranger with their children. So, KibaruaNow decided to focus on housecleaning.
Each day new taskers were sent to clean homes, and many youth earned two or three times more than they could before. The business started to grow. There was one problem. KibaruaNow was losing money on every transaction and headed towards a financial cliff. Although the team had agreed on metrics for success and was given clear direction to stay small until the unit economics were validated, no one on the team could bring themselves to turn away the youth who arrived in desperate need of work. More and more effort became focused on running the business rather than experimenting with ways to bring costs down and improve revenues. We never landed on a viable business model and ultimately had to shut down the enterprise.
Oftentimes the pressure to grow is not internal, but external. Most funders want to see tangible results for the money they provide, and that means numbers – the bigger the better. The websites of nonprofits, social enterprises, foundations, government agencies, and impact investors are littered with vanity metrics on the number of people, communities, schools, or small businesses that have been reached. Yet such raw numbers only indicate activity, not whether the intervention was effective.
Some organizations may go further to include aggregate measures of benefit, such as lives saved, kids educated, or incomes increased. This is a significant improvement as we can attribute a positive effect. But, does it merely reflect prolific fundraising? Could another entity have made a far greater impact with those same funds? I once came across a program that claimed to have raised the aggregate incomes of smallholder farmers in a community by 1 million dollars, which sounded fantastic until I discovered that 10 million dollars had been spent to do so.
Harsh financial realities can also cause a premature shift from validation to delivery. Nonprofits and social enterprises operate on a shoestring. Sometimes accepting a strategically misaligned grant can mean the difference between making payroll or layoffs. There is no shame in this. But don’t let yourself be perpetually caught up in funding cycles and donor priorities, and lose sight of your ultimate goal. If you can explicitly acknowledge when you decide to make a temporary compromise and keep it within tight constraints, you can map a path back to your vision and the validation required to get there.
VALUE, GROWTH, AND IMPACT
In the business world, recognizing the signs of success is simple. If users love a product or service, they will buy it, tell their friends, and come back for more. Because the user of a product or service is typically also the purchaser, assumptions regarding value and growth tend to be well aligned. As a result, most commercial design practices focus on validating customer value.
The job of a mission‐oriented organization is far more complex. Not only do value and growth frequently involve different customers with divergent priorities, but social impact, rather than profits, is the true goal. Thus, a successful solution must fulfill all three of the sometimes conflicting pillars of social innovation: value, growth, and impact (see Figure 6.1).
Figure 6.1 The three pillars of social innovation.
We typically start with value. After all, if we don’t deliver value to our beneficiaries, our solution may not be used, bought, or recommended to others. When evaluating value, what people do is far more revealing than what they say. If someone is asked a hypothetical question about whether they like a product or service, a positive response may merely indicate a reluctance to offend. But if users are beating a path to your door and bringing their friends, you’re likely onto something. The more your users find value in what you are offering, the easier it will be to drive growth and ultimately impact.
In concert with value, we also need to consider the engine that can accelerate growth over time. Too often, interventions are scaled through brute force with grant money, and sustainable models for scale are only evaluated after the limits of charitable funding are exhausted. The problem is that injecting a new growth strategy can have significant implications that may force you to completely redesign your solution. For a market‐driven business model, you may need to target a lower price point based on ability to pay. Replication or franchising will only work with a simple, well‐defined intervention that can be delivered equally well by a third party with far less training and expertise. Or if deploying through government, your costs and processes must conform to existing budgets, policies, and politics. By testing the engine for growth early, your solution can evolve in ways that will enable it to scale over time.
Of course, our ultimate goal is to deliver social impact. People may demand potato chips and you might have a compelling business plan to sell them, but what social benefit does it offer? The impact hypothesis can often be the most challenging to validate, as conclusive impact may take years to prove – think of challenges such as increasing high school graduation rates, reducing recidivism, combating climate change, or ending the cycle of violence or poverty. Yet, lighter‐weight proxies can often give an indication of whether you are on track sooner and help you to refine your approach so that success is more likely down the line. Compelling impact will also increase perceived value, further fueling adoption.
What does it look like when one of these three pillars is missing? If users don’t perceive sufficient value and thus don’t want or demand a solution, you might find yourself in the same situation as those trying to provide polio vaccinations in Nigeria and Pakistan. Though the vaccine has proven effective and is funded for worldwide administration, some families refuse treatment due to religious beliefs, false rumors of health risks, or distrust of workers. A similar lack of perceived value has at times hampered clean cookstoves, toilets, mosquito nets, and numerous other interventions from being used as intended.
If no viable path exists to accelerate growth, the result might resemble the first incarnation of EARN, a nonprofit helping low‐income Americans save money that we learned about in Chapter Two, when a highly effective and desirable solution was only reaching a tiny fraction of those in need. This is all too common in the social sector, where the scale of most solutions begins to plateau far short of the need. Not only does this leave out many who suffer, but the overall value for money is significantly lower without economies of scale.
And if the intended social impact does not materialize, an intervention might become widespread without delivering the promised benefit. As we’ll learn about more in Chapter Nine, in the case of microfinance decades passed and hundreds of millions of customers were reached before rigorous evaluations contradicted the original claims of increased incomes and women’s empowerment. The fascinating book Poor Economics, by Abhijit Banerjee and Esther Duflo, chronicles numerous such failures in the global development arena.2
Few mission‐driven organizations embrace and validate all three pillars of social innovation from the start. As a consequence, precious time, money, and even lives are being wasted. This is one reason the sector as a whole vastly underperforms relative to its potential. By investing in more diligence upfront to ensure solutions meet real user needs, have a sustainable engine for growth, and achieve the desired social impact, far more social benefit can be created over time.
Some trailblazing entities have stepped in with tools and techniques to deliver on value, growth, or impact. Each has their aficionados and advocates, along with associated funding, luminaries, and consultants. Yet, this expertise largely remains siloed and rarely are all three considered in concert. At the forefront is value, or desirability, which is moving into the mainstream, driven by HCD and behavioral science. The importance of evaluating impact continues to gain traction, bolstered by development economists, the effective altruism movement, and a collection of evidence‐first nonprofits and funders, although it is not often integrated into an iterative learning process. Perhaps least prevalent are growth models that enable solutions to reach true scale, with leadership coming from social entrepreneurs exploring innovative business models, some larger nonprofits engaging with governments, and others (such as the Skoll Foundation) who promote systems change.
As the next frontier, Lean Impact proposes a holistic approach that incorporates value, growth, and impact from the start. There are no shortcuts. To realize social impact at scale, we need to deliver on all three.
STAGING RISK
How do you determine which assumptions to test first? There’s no simple formula, but it is important to consider risk, time, and cost. The goal is to eliminate the greatest degree of risk with the least investment of time and money (see Figure 6.2). A good place to start is by identifying the killer assumptions – the biggest risks with the potential to make or break your solution. Both internal and external skeptics may have something to say on this topic. Give them a chance to voice their concerns.
Figure 6.2 Prioritizing assumptions for Tenofovir (illustrative only).
Your killer assumptions can each be broken down into one or more hypotheses, which are in turn tested using one or more MVPs. With a dose of creativity, an MVP can be quite simple. Consider the roughest prototype, fewest people, and fastest experiment that could help you learn. While an individual test might only shed light on one dimension of an assumption, it may provide sufficient insight to immediately invalidate a particular path or give you the confidence to proceed with more expensive tests.
Of course, some assumptions realistically take more time to test than others. Long‐term impacts, such as educational attainment, breaking the cycle of poverty, or improved health may take years to fully manifest. And, obtaining rigorous evidence of such impact through randomized control trials (RCTs) or similar tools can be slow and costly. But that doesn’t let you off the hook to learn as much as possible first. If you find that your killer assumptions are landing in the upper‐left corner of Figure 6.2, look for ways to break them down into early indicators that are predictive of those eventual outcomes and can be tested more cheaply and quickly. We’ll explore this more when we delve into the impact hypothesis in Chapter Nine.
In certain situations – such as for medical drugs, devices, and procedures – regulations may even require that an RCT be completed before a solution can be deployed. But that doesn’t mean we should plow forward blindly. Consider the case of Tenofovir, a vaginal gel intended to prevent the transmission of HIV. Through early clinical trials in a highly‐controlled environment, it was found to be safe to use and effective in killing the virus. Given these promising results, large aid donors paid millions of dollars for a three‐year phase‐3 clinical trial called FACTS 001 that was undertaken in South Africa as the final step for regulatory approval. The result? No statistically significant difference between the placebo group and the treatment group.3 How could that be?
It turned out that the women in the trial didn’t use the gel consistently both before and after every sexual encounter as required. While Tenofovir worked in a controlled clinical setting, the messy reality of life made it undesirable and impractical. Given cultural dynamics, stopping to apply the gel in the heat of the moment often wasn’t possible. Of course, skipping the phase‐3 trial was not an option. But a much shorter and cheaper test could have been conducted first to understand the women’s preferences and realities – verifying value before impact. If that had been done, the researchers might have redesigned or abandoned the drug before investing in such a big trial.
The order of testing can be just as important as which assumptions are tested. Start as small as possible with the biggest risks. Then add more complex, expensive experiments after eliminating the more obvious ones.
FASTER ITERATION
Once you have identified the first assumption to test, your validation cycle begins. For each iteration, start with an assumption, formulate a hypothesis to validate or invalidate the assumption, build an MVP to test that hypothesis, run experiments to measure the response, and learn whether your hypothesis is in fact true. Even if the initial results are positive, you may want to consider additional dimensions, different conditions, or a larger sample size. Or, if you have gained sufficient confidence, it may be time to move onto the next riskiest assumption. On the other hand, if the results are negative, take a step back and consider whether to refine your test, improve your solution, or make a more significant pivot.
The most crucial factor for success is not designing the perfect experiment, but rather the speed at which you can execute each turn of your feedback loop. A cycle is the basic unit of learning. When we don’t consolidate learning until months or years into a program, we miss opportunities to course correct along the way. Imagine sailing a boat without checking the direction, speed, location, and wind regularly. You’d be unlikely to take an efficient route. By reducing the time for each iteration, we can learn and improve more quickly.
Prototypes and feedback loops are easy if you’re running an online service. It’s no coincidence that the accelerated pace of progress in Silicon Valley corresponds to the transition from boxed software for personal computers to online services in the cloud. Companies such as Google and Facebook are able to deploy hundreds of A/B tests every day as controlled experiments to compare the performance of their current service (the control, or A) with a new version of their user interface or algorithm (the variation, or B). Some of these may be minor tweaks in layout, language, or color. Others may include new features, modified algorithms, or more dramatic redesigns of the product or service. Today, almost all online sites stage their rollouts by first running A/B tests with a smaller cohort to ensure that any change works well and represents a real improvement.
Of course, many, if not most, social innovations don’t involve an online service. Realistically, this means tests will require more manual effort and not be quite so instantaneous. But the same principles apply. Iteration time can be reduced by starting with a small cohort and designing the simplest MVP that will shed light on a hypothesis. Summit Public Schools reconfigured a classroom for a week, Nexleaf Analytics tried different cookstove designs with a small number of women, and the One Acre Fund now tests new crops with a few farmers first.
When Kudoz first prototyped its service, it asked 20 adults with a cognitive disability to select from options in a mocked‐up catalog. Within a week, it delivered the first experiences by having existing team members serve as hosts. As most of its clients were nonverbal, it created a range of visual and tactile materials that could be used to self‐identify emotions before and after an experience and took photos to look at body language. This allowed Kudoz to measure changes in motivation, capabilities, and engagement to determine how its offerings were being received and whether it was having a positive effect.
In the following three chapters, I’ll share numerous other examples of how organizations tested their value, growth, and impact hypotheses.
COLLECTING DATA
Through their extensive work with early‐stage social entrepreneurs, the venture philanthropists at the Draper Richards Kaplan Foundation have found that a culture of rigorous data collection is one of the primary drivers for greater impact. Good, timely data gives us an unbiased view into experiments so that decisions can be made scientifically rather than be based on potentially biased opinions. It can also be used to monitor ongoing performance, drive continuous improvements, and quickly surface any issues.
Although my own background has been in the tech sector, I’ve tended to shy away from leaping to technology‐based solutions for global development challenges. In low‐income countries, too often basic access or literacy doesn’t exist. And a cool gadget can divert you from first understanding the underlying need. I have generally found it more fruitful to start with an analog solution, iterating on it to convincingly understand a problem, and only then consider digital automation as a means to lower distribution costs and increase scale.
Where I have found technology to be a consistent game changer, however, is for data collection. There simply is no substitute for the speed, accuracy, and flexibility of capturing, sharing, and analyzing data digitally. During the Ebola outbreak in Liberia, one of the biggest challenges was obtaining accurate information on transmission rates, geographic spread, and health service availability. A member of my team at USAID who was deployed to the country found that data was often collected on scraps of paper, transmitted by motorbike, and entered multiple times into incompatible health information systems. A slow and error‐prone process.
Timely, accurate data is the fuel that drives your feedback loop. The purpose of experiments and MVPs is not the prototype itself, but rather the resulting data that tells you what works and what doesn’t work so you can learn, adapt, and improve.
In Chapter One, we learned how Summit Public Schools uses a technology platform to manage curriculum, assessments, and student information. By analyzing this data, Summit gains rapid insight into what is working or not, so they can quickly adapt. While most attention focuses on the transformative potential of user‐facing technologies, such enterprise data systems can make a huge difference in accelerating innovation and improving results.
Given the proliferation of mobile phones and the increasing sophistication of big data tools, collecting and analyzing data digitally has become feasible almost everywhere. When Living Goods was founded in 2007 to improve rural healthcare through a network of Avon‐like health entrepreneurs, only 30% of the Ugandan population had mobile phones. Five years later, with mobile penetration at 70%, the organization reinvented itself to be digital first. Founder Chuck Slaughter found the shift transformative, moving “data at light speed versus bicycle speed.” Using digital tools, Living Goods has been able to reduce costs, improve accuracy, and decrease the turnaround time of getting data back from the field, thus dramatically increasing its pace of learning.
One experiment with 30 community health promoters tested a prototype Android app built by two Kenyan programmers for under $2000 to register pregnant mothers. Using the streamlined user interface of this new smartphone app, health workers dramatically increased the number of pregnancies they accurately identified and captured from 35% to 85%. This early identification enables Living Goods to reduce complications by providing health and nutrition tips, flagging risk factors, monitoring danger signs, and making referrals to a health facility if necessary.
The decreasing cost and increasing sophistication of sensor technologies and the Internet of Things opens up new possibilities for low‐cost data gathering. Nexleaf Analytics is building cloud‐based sensors to remotely monitor cold‐chain storage of vaccines and usage of clean cookstoves. Smaller steps can also be taken with technology‐enabled solutions that are not fully automated. In refugee camps in across Africa, the American Refugee Committee (ARC) has deployed a real‐time feedback system called Kuja Kuja to track customer satisfaction with water distribution, healthcare, and other services. Refugees employed by ARC stand at service locations with mobile‐enabled tablets and ask two simple questions: Are you satisfied with the service, and do you have an idea to make us better? The results are shared transparently on a public dashboard.
A common complaint for many programs is the lack of data on outcomes beyond the end of an engagement. By following up via text messages every four months for two years, Harambee Youth Employment Accelerator has been able to keep its fingers on the pulse of the trajectory for its job seekers in South Africa. Youth report whether they secured employment, were promoted, lost their job, or found Harambee helpful in finding a job. With a 30% (and growing) response rate, it’s been able to glean significant insights it can use to tune its products and services.
As organizations collect, collate, and analyze more and more data, new possibilities will open for analytical tools to better predict trends, understand correlations, and identify opportunities. Perhaps one day, the same algorithms that Amazon uses to predict the next product you will want to purchase will be used to predict what intervention is most likely to transform someone’s life for the better.
SUCCESS CRITERIA
Objective success criteria, established before experiments are launched, are important to counterbalance the groupthink that can arise from enthusiasm or exhaustion. Determine these targets based on what will be necessary to reach your goal rather than what seems possible for you to achieve. This will typically reflect a combination of appreciable improvement relative to the status quo, a cost structure that can be scaled, and a realistic assessment of how the pieces come together to deliver the intended outcome. How many doors do you have to knock on to get a positive response? What percentage of recipients need to correctly use the goods or training to improve health, reduce crime, or protect the environment? How often do people come back for more or bring their friends?
It is important to note that each of these innovation metrics is based on unit measures. That is, for each 100 attempts, how frequently does the desired result occur? In contrast, vanity metrics typically report on absolute numbers and are easy to game simply by pouring in more dollars, time, or people. Table 6.1 illustrates a simplified set of success criteria for a hypothetical intervention to improve smallholder farmer incomes. One way to reach 50% of all farmers in a region could be to hire sales staff to do outreach. Perhaps the pitch will be compelling enough for 30% of the farmers to sign up right away, and the remaining 20% might come later through referrals. To be financially sustainable and allow for growth, we might seek a small net profit from each farmer who participates. They’ll need to be willing to pay more than our marginal costs, and if we offer loans we should also factor in default rates. Finally, if the overall goal is for average income to increase by 70%, perhaps crop yields will need to increase by 90%, to allow for some spoilage or dysfunctional markets.
Table 6.1 Success criteria example.
| Success criteria | Hypotheses (to test) | Target |
| Value: 50% of all farmers in the region will participate | Percentage of farmers approached by sales staff who will sign up | 30% |
| Average number of referrals after first growing season | 2 | |
| Growth: Net contribution margin of $10 per farmer | Amount farmers are willing to pay for seeds, fertilizer, and training | $200 |
| Percentage loan repayment rate | 95% | |
| Cost of raw materials and staff | ‐$180 | |
| Impact: Average income increases by 70% for participating farmers | Average percentage increase in crop yields | 90% |
| Percentage of crops spoiled or unsold | 20% |
Notice that each of these discretely measurable hypotheses can be validated through experiments. If they all prove to be true, then the associated success criteria will likely be met. If not, the strategy may need to be adjusted. For example, if only 20% of farmers respond to recruiters initially, the success criteria of 50% participation could still be achieved if each then refers a larger number of friends. Similarly, higher costs or default rates could work if greater revenues can be generated. And if markets or transport options prove to be highly unreliable, even higher yield increases may be required. Throughout the validation process, keep an eye on your success criteria, make adjustments based on your results, and recognize if all likely paths become blocked and it’s time to pivot.
DIMINISHING RETURNS
For validation, starting small is essential. But you also don’t want to only run experiments forever. Certainly, if you aren’t making good headway towards your success criteria for value, growth, and impact, it may be time to consider a pivot. However, if you have successfully validated your killer assumptions, continuing to test every last hypothesis will soon run into the law of diminishing returns.
Don’t let yourself get too comfortable. Remember, the reason we want to start small is to spend the least amount of time and money learning the lessons we need to learn. It’s a waste to fail with 1000 when we can fail with 10. It is also true that as our remaining assumptions become less and less risky, running lots of small experiments, even cheaply, can be a different waste of precious time and money. We want to always sit on the edge of comfort, so that we continue to learn as fast as possible. That means that once we have gained reasonable confidence, we should aggressively take on more risk in the form of greater scale, a more realistic deployment, or new contexts.
The goal is not to validate a solution to 100% confidence. That will never happen. The goal is to answer the make‐or‐break questions and to do so with the smallest investment possible.