Why Our Sample Sizes Are Too Small
Let’s say you want to hire a secretary (sorry: a PA). A hundred women have applied for the role, and you’re interviewing them one by one in random order. After each interview, you have to make a decision: will I hire or reject this candidate? No sleeping on it, no putting it off until you’ve seen all the applicants. The decision you make straight after the interview cannot be overridden. How do you proceed?
Do you take the first candidate who makes a decent impression? If so, then you run the risk of missing out on the best candidates, because you’d probably meet applicants who are just as good or even better further down the line. Or do you interview the first ninety-five to get a feel for the quality of the pool, then choose the one from the final five who’s most similar to the best candidate you’ve seen so far? And what if the final five are all disappointments?
This question is known among mathematicians by a politically incorrect label: the secretary problem. Surprisingly, there is only one optimal solution. You should interview the first thirty-seven candidates and reject them all; meanwhile, however, you should be monitoring their quality. Then keep interviewing until you find someone who is better than the top applicant out of the previous thirty-seven. Hire her. You’ll be making an excellent decision. She may not be the very best of the hundred applicants, but she’s sure to be a solid choice. Every other approach has been shown to produce statistically worse results.
What is it about the number thirty-seven? Thirty-seven is 100 divided by the mathematical constant e (2.718). If you had only fifty applicants, you would turn down the first eighteen (50/e) then hire the first candidate who was better than anyone out of the previous eighteen.
Originally, the secretary problem was also known as the marriage problem, and the question was this: how many potential partners should I “try out” before I marry one? But because the total number of possible life partners can’t be known in advance, the approach described above isn’t ideal. Hence why the mathematicians renamed the problem.
Now, the good life isn’t a question of mathematical exactness. As Warren Buffett says, “It’s better to be approximately right than precisely wrong.” This is how Buffett makes investment decisions; you should take the same tack in your personal life. So why is the secretary problem still relevant? Because it gives us some guidelines about how long we should spend testing things out before we make a final decision on important issues. Experiments with the secretary problem have shown that most people plump for a candidate too soon. Try to resist this impulse. When it comes time to pick a career, a job, an industry, a partner, a place to live, a favorite author, a musical instrument, a preferred sport or an ideal holiday destination, it’s worthwhile quickly trying out many different options at first—more options than you’d like—before making a firm decision. Choosing before you have a strong sense of what’s out there is not a sensible idea.
Why do we tend to make decisions too soon? Where does our impatience come from? Random sampling is time-intensive. Doing a hundred job interviews when we could be done in five? Going through ten different application procedures before picking a place to work? It’s a lot of effort—much more than we’d like. Plus, sampling is gluey. It’s easy to get stuck in an industry simply because you poked around in it a bit when you were young. You might have built a career in it, of course, but you almost certainly would have met with the same success elsewhere, or perhaps with more success and greater enjoyment—if only you’d been a little more willing to experiment. The third reason why we tend to make choices too quickly is that we prefer to keep our minds clear. We like ticking stuff off and moving onto the next item on the list. That’s fine for inconsequential decisions, but counterproductive for the important ones.
Our nanny (aged twenty) knocked on our door a few months ago looking rather dispirited. Her first and thus far only boyfriend had dumped her. Her eyes were brimming with tears. We tried being rational and level-headed: “You’re still so young, you’ve got so much time! Try out ten or twenty men. Then you’ll know what’s on the market. You’ll find out who’s really suited to you long-term, and who you’re suited to.” A weak smile appeared on her gloomy face. I don’t think we were able to convince her—not then, anyway.
Unfortunately, we behave far too often like my nanny. Our sample sizes are too small, our decisions rushed—or, in the language of statisticians, not representative. We rely on a false impression of reality, believing that with a few random spot tests we can find the man or woman of our dreams, our ideal job, the best place to live. Sure, it might work out—if so, then I’m thrilled for you—but if it does, it will only be by a stroke of good fortune, and nobody should pin their hopes on that.
The world is much bigger, richer and more diverse than we imagine, so try to take as many samples as you can while you’re still young. Your first years of adulthood aren’t about earning money or building a career. They’re about getting acquainted with the universe of possibility. Be extremely receptive. Taste whatever fate dishes up. Read widely, because novels and short stories are excellent simulations of life. Only as you age should you adapt your modus operandi and become highly selective. By then you’ll know what you like and what you don’t.