NOTES

INTRODUCTION: THINKING, CRITICALLY

avoid learning a whole lot of things that aren’t so: After Huff, D. (1954/1993). How to Lie with Statistics. New York: W.W. Norton, p. 19. And, as you’ll read later, he probably was echoing Mark Twain, or Josh Billings, or Will Rogers, or who knows who.

Misinformation has been a fixture of human life: Abraham provides misinformation about the identity of his wife, Sarah, to King Abimelech to protect himself. The Trojan horse was a kind of misinformation, appearing as a gift but containing soldiers.

PART ONE: EVALUATING NUMBERS

People choose what to count: This sentence is nearly a direct quote from Best, J. (2005). Lies, calculations and constructions: beyond How to Lie with Statistics. Statistical Science, 20(3), 210–214.

More people have cell phones than toilets: Wang, Y. (2013, March 25). More people have cell phones than toilets, U.N. study shows. http://news feed.time.com/2013/03/25/more-people-have-cell-phones-than-toilets-u-n-study-shows/.

150,000 girls and young women die of anorexia each year: Steinem, G. (1992). Revolution from Within. New York: Little, Brown. Wolf, N. (1991). The Beauty Myth. New York: William Morrow.

Add in women from twenty-five to forty-four and you still only get 55,000: This example came to my attention from Best, J. (2005). Lies, calculations and constructions: beyond How to Lie with Statistics. Statistical Science, 20(3), 210–214. The statistics are available at www.cdc.gov.

anorexia deaths in one year cannot be three times the number of all deaths: Maybe you’re in the accounts payable department of a big corporation. An employee put in for reimbursement of gasoline for the business use of his car, $5,000 for the month of April. Start with a little world knowledge: Most cars get better than twenty miles per gallon these days (some get several times that). You also know that the fastest you can reasonably drive is seventy miles per hour, and that if you were to drive ten hours a day, all on the freeway, that would mean 700 miles a day. Keep that up for a standard 21.5-day work month and you’ve got 15,050 miles. In these kinds of rough estimates, it’s standard to use round numbers to make things easier, so let’s call that 15,000. Divide that by the fuel economy of 20 mpg and, by a rough estimate, your employee needed 750 gallons of gas. You look up the average national gas price for April and find that it’s $2.89. Let’s just call that $3.00 (again, rounding, and giving your employee the benefit of the doubt—he may not have managed to get the very best price every time he filled up). $3/gallon times 750 gallons = $2,250. The $5,000 on the expense report doesn’t look even remotely plausible now. Even if your employee drove twenty hours a day, the cost wouldn’t be that high. https://www.fueleconomy.gov/feg/best/bestworstNF.shtml, retrieved August 1, 2015. http://www.fuelgaugereport.com/.

a telephone call has decreased by 12,000 percent: Pollack, L., & Weiss, H. (1984). Communication satellites: countdown for Intelsat VI. Science, 223(4636), 553.

one of 12,000 percent seems wildly unlikely: I suppose you could spin a story that makes this true. Maybe a widget used to cost $1, and now, as part of a big promotion, a company is not just willing to give it to you for free, but to pay you $11,999 to take it (that’s a 12,000 percent reduction). This happens in real estate and big business. Maybe an old run-down house needs to be razed before a new one can be built; the owner may be paying huge property taxes, the cost of tearing down the house is high, and so the owner is willing to pay someone to take it off of his or her hands. At one point in the late 1990s, several large, debt-ridden record companies were “selling” for $0, provided the new owner would assume their debt.

200 percent reduction in customer complaints: Bailey, C., & Clarke, M. (2008). Aligning business leadership development with business needs: the value of discrimination. Journal of Management Development, 27(9), 912–934.

Other examples of a 200 percent reduction: Rajashekar, B. S., & Kalappa, V. P. (2006). Effects of planting seasons on seed yield & quality of tomato varieties resistant to leaf curl virus. Seed Research, 34(2), 223–225. http://www.bostoncio.com/AboutRichardCohen.asp.

50 percent reduction in salary: Illustration © 2016 by Dan Piraro based on an example from Huff, ibid.

making this distinction between percentage point and percentages clear: I’m grateful to James P. Scanlan, attorney-at-law, Washington, D.C., who answered my query to the membership of the American Statistical Association, and provided me with this misuse.

closing of a Connecticut textile mill and its move to Virginia: This example comes from Spirer, L., Spirer, H. F., & Jaffe, A. J. (1987). Misused Statistics, New York: Marcel Dekker, p. 194.

Miller, J. (1996, Dec. 29). High costs are blamed for the loss of a mill. New York Times, Connecticut Section.

And n. a. (1997, Jan. 12). Correction, New York Times, Connecticut Section.

legislation that denied additional benefits: McLarin, K. J. (1993, Dec. 5). New Jersey welfare’s give and take; mothers get college aid, but no extra cash for newborns. New York Times.

See also: Henneberger, M. (1995, April 11). Rethinking welfare: deterring new births—a special report; state aid is capped, but to what effect? New York Times.

births to welfare mothers had already fallen by 16 percent: Ibid.

no reason to report the new births: Ibid.

Although they are mathematically equivalent: Koehler, J. J. (2001). The psychology of numbers in the courtroom: how to make DNA-match statistics seem impressive or insufficient. Southern California Law Review, 74, 1275–1305.

And Koehler, J. J. (2001). When are people persuaded by DNA match statistics? Law and Human Behavior, 25(5), 493–513.

On average, humans have one testicle: Attributed to mathematics professor Desmond MacHale of University College, Cork, Ireland.

temperatures ranging from 15 degrees to 134 degrees: http://en.wikipedia.org/wiki/Death_Valley.

the amount of money spent on lunches in a week: As an example, suppose six adults spend the following amounts on lunch {$12, $10, $10, $12, $11, $11} and six children spend the following {$4, $3.85, $4.15, $3.50, $4.50, $4}. The median (for an even number of observations, the median is sometimes taken as the mean between the two middle numbers, or in this case, the mean of 4.5 and 10) is $7.25. The mean and median are amounts that no one actually spends.

During the 2004 U.S. presidential election: See Gelman, A. (2008). Red State, Blue State, Rich State, Poor State. Princeton, NJ: Princeton University Press.

the average life expectancy for males and females: These numbers are for white males and females. Non-white figures for 1850 are not as readily available. http://www.infoplease.com/ipa/A0005140.html. An additional source of concern is that the U.S. numbers for 1850 are for the state of Massachusetts only, according to the Bureau of the Census.

the average family: The title of this section, and the discussion, follows the work of Jenkins and Tuten very closely:

Jenkins, J., & Tuten, J. (1992). Why isn’t the average child from the average family? And similar puzzles. American Journal of Psychology, 105(4), 517–526.

the average number of siblings: Stick-figure children from Etsy, https://www.etsy.com/listing/221530596/stick-figure-family-car-van-bike-funny; small and large house drawn by the author; medium house from http://www.clipartbest.com/clipart-9TRgq8pac.

average investor does not earn the average return: A simulation, see Tabarrok, A. (2014, July 11). Average stock market returns aren’t average. http://marginalrevolution.com/marginalrevolution/2014/07/average-stock-market-returns-arent-average.html. Accessed October 14, 2014.

poster presented at a conference by a student researcher: Tully, L. M., Lincoln, S. H., Wright, T., & Hooker, C. I. (2013). Neural mechanisms supporting the cognitive control of emotional information in schizophrenia. Poster presented at the 25th Annual Meeting of the Society for Research in Psychopathology. https://www.researchgate.net/publication/266159520_Neural_mechanisms_supporting_the_cognitive_control_of_emotional_information_in_schizophrenia.

I first found this example at www.betterposters.blogspot.com.

gross sales of a publishing company: http://pelgranepress.com/index.php/tag/biz/.

Fox News broadcast the following graph: I’ve redrawn this for the sake of clarity. For the original, see http://cloudfront.mediamatters.org/static/images/item/fbn-cavuto-20120731-bushexpire.jpg.

Discontinuity in vertical or horizontal axis: Spirer, Spirer, & Jaffe, op. cit., pp. 82–84.

Choosing the proper scale and axis: Example from Spirer, Spirer, & Jaffe, op. cit., p. 78.

Many things change at a constant rate: Spirer, Spirer, & Jaffe, op. cit., p. 78.

life expectancy of smokers versus nonsmokers at age twenty-five: These data taken from Jha, P., et al. (2013). 21st-century hazards of smoking and benefits of cessation in the United States. New England Journal of Medicine, 368(4), 341–350, Figure 2A for women. Survival probabilities were scaled from the National Health Interview Survey to the U.S. rates of death from all causes at these ages for 2004 with adjustment for differences in age, educational level, alcohol consumption, and adiposity (body-mass index). I’m grateful to Prabhat Jha for her correspondence about interpreting this.

This form of presentation is based on that of Wainer, H. (1997). Visual Revelations: Graphical Tales of Fate and Deception from Napoleon Bonaparte to Ross Perot. New York: Copernicus/Springer-Verlag.

expenditures per public school student and those students’ scores on the SAT: This example from Wainer, H. (1997). Visual Revelations: Graphical Tales of Fate and Deception from Napoleon Bonaparte to Ross Perot. New York: Copernicus/Springer-Verlag, p. 93. The original appeared in Forbes (May 14, 1990).

Of course, there are other variables. Are the spending increases reported in actual or inflation-adjusted dollars? Was the time frame 1980–88 chosen to make that point, and would a different time frame make a different point?

The correlation also provides a good estimate: There is some controversy about whether to use r or r-squared. For the defense of r, see: D’Andrade, R., & Dart, J. (1990). The interpretation of r versus r2 or why percent of variance accounted for is a poor measure of size of effect. Journal of Quantitative Anthropology, 2, 47–59.

Ozer, D. J. (1985). Correlation and the coefficient of determination. Psychological Bulletin, 97(2), 307–315.

services provided by the organization Planned Parenthood: Roth, Z. (2015, Sept. 29). Congressman uses misleading graph to smear Planned Parenthood. msnbc.com.

Politifact explored this issue further, examining the data between the endpoints and furnishing additional contextual information to go along with the usual graph-centered criticism. See https://perma.cc/P8NY-YP49.

presentation on iPhone sales: http://qz.com/122921/the-chart-tim-cook-doesnt-want-you-to-see/; http://www.tekrevue.com/tim-cook-trying-prove-meaningless-chart/.

feature spurious co-occurrences: http://www.tylervigen.com/spurious-correlations.

Randall Munroe in his Internet cartoon xkcd: https://xkcd.com/552/.

visual system is pitted against your logical system: This example is based on one in Huff, ibid.

Any model of consumer behavior on a website: This is nearly a direct quote from De Veaux, R. D., & Hand, D. J. (2005). How to lie with bad data. Statistical Science, 20(3), 231–238, p. 232.

Colgate’s biggest competitor was named nearly as often: I thank my student Vivian Gu for this example.

Derbyshire, D. (2007, Jan. 17). Colgate gets the brush off for “misleading” ads. The Telegraph. Retrieved from http://www.telegraph.co.uk/news/uknews/1539715/Colgate-gets-the-brush-off-for-misleading-ads.html.

C-SPAN advertises that it is “available”: http://www.c-span.org/about/history/.

doesn’t mean that even one person is watching: Nielsen reports that Americans, on average, receive 189 channels but watch only 17 of them. http://www.nielsen.com/us/en/insights/news/2014/changing-channels-americans-view-just-17-channels-despite-record-number-to-choose-from.html.

water use in the city of Rancho Santa Fe: Boxall, B. (2014, Dec. 2). Rancho Santa Fe ranked as state’s largest residential water hog. Los Angeles Times. http://www.latimes.com/local/california/la-me-water-rancho-20141202-story.html.

Lovett, I. (2014, Nov. 29). “Where grass is greener, a push to share drought’s burden.” New York Times. http://www.nytimes.com/2014/11/30/us/where-grass-is-greener-a-push-to-share-droughts-burden.html.

flying is actually safer now: http://www.flightsafety.org; Grant, K. B. (2014, Dec. 30). Deadly year for flying—but safer than ever. http://www.cnbc.com/id/102301598.

Newton’s law of cooling: For an initial temperature of 155 degrees Fahrenheit, the formula is

f(t) = 80e−0.08t + 75.

C-SPAN is available in 100 million homes: Bedard, P. (2010, June 22). “Brian Lamb: C-SPAN now reaches 100 million homes.” U.S. News & World Report. www.usnews.com/news/blogs/washington-whispers/2010/06/22/brian-lamb-c-span-now-reaches-100-million-homes. Retrieved November 22, 2010.

90 percent of the population is within twenty-five miles: Based on Huff, op. cit., p. 48.

3,482 active-duty U.S. military personnel who died in 2010: https://www.cbo.gov/sites/default/Files/113th-congress-2013-2014/workingpaper/49837-Casualties_WorkingPaper-2014-08.pdf.

total of 1,431,000 people in the military: http://www.census.gov/compendia/statab/2012/tables/12s0511.pdf.

death rate in 2010: http://www.cdc.gov/nchs/fastats/deaths.htm.

general population of the United States includes: Based on an example from Huff, op. cit., p. 83.

increase in the number of doctors: I thank my student Alexandra Ghelerter for this example. Barnett, A. (1994). How numbers are tricking you. Retrieved from http://www.sandiego.edu/statpage/barnett.htm.

nuances often tell a story: This is Best’s term.

there are six different indexes: Davidson, A. (2015, July 1). The economy’s missing metrics. New York Times Magazine.

July 2015 that the unemployment rate dropped: Shell, A. (2015, July 2). Wall Street weighs Fed’s next move after jobs data. USA Today Money. http://americasmarkets.usatoday.com/2015/07/02/wall-street-gets-what-it-wants-in-june-jobs-count/.

reported the reason for the apparent drop: Schwartz, N. D. (2015, July 3). Jobless rate fell in June, with wages staying flat. New York Times, B1.

batting averages for the 2015 season: Stats from http://mlb.mlb.com/stats/sortable.jsp#elem=[object+Object]&tab_level=child&click_text=Sortable+Player+hitting&game_type=%27R%27&season=2015&season_type=ANY &league_code=%27MLB%27&sectionType=sp&statType=hitting&page=1&ts=1457286793822&playerType=QUALIFIER&timeframe=.

top three causes of death in 2013: http://www.cdc.gov/nchs/fastats/leading-causes-of-death.htm.

attitudes do not seem to fall upon racial lines: This is entirely hypothetical.

Another hurdle: You want age variability: This is from Huff, op. cit., p. 22.

71 percent of which British?: Ibid.

answer falsely just to shock the pollster: Many years ago, Chicago columnist Mike Royko encouraged readers to lie to exit pollers on Election Day in the hope that inaccurate data and being made to look foolish would end the practice of TV commentators calling the result of an election before all the votes were counted. I have no data on how many people lied to the exit pollers because of Royko’s column, but the fact that exit polls are still a thing suggests it wasn’t enough.

the price you pay for not hearing from everyone: Taken from http://www.aapor.org/AAPORKentico/Education-Resources/For-Researchers/Poll-Survey-FAQ/What-is-the-Margin-of-Sampling-Error.aspx.

Note that these ranges overlap: This is a good rule of thumb, but in some cases this quick method will be inaccurate. See Schenker, N., & Gentleman, J. F. (2001). On judging the significance of differences by examining the overlap between confidence intervals. American Statistician, 55(3), 182–186.

Five times out of a hundred: I’m intentionally not making a distinction here between frequentist and Bayesian probability estimates, a distinction that comes up in Part Two.

Margin of error: (image) From Wikipedia.

formula for calculating the margin of error: For large populations, the 95 percent confidence interval can be estimated as ±1.96 × sqrt [p(1-p)/n]. To obtain a 99 percent confidence interval, multiply by 2.58 instead of 1.96. Yes, the interval is larger when you’re more confident (which should make sense; if you want to be more sure that the range you quote includes the true value, you need a larger range). For smaller populations, the formula is to first compute the standard error:

sqrt [{(Observed proportion) × [1 – (Observed proportion)}/sample size]

The width of the 95 percent confidence interval then is ±2 × standard error.

For example, if you sampled fifty overpasses in a large city, you might have found that 20 percent of them needed repair. You calculate the standard error as:

sqrt [(.2 x .8)/50] = sqrt (.16/50) = .057.

So the width of your 95 percent confidence interval is ±2 × .057 = ±.11 or ±11%. Thus the 95 percent confidence interval is that 20 percent of the overpasses in this town need repair, plus or minus 11 percent. In a news report, the reporter might say that the survey showed 20 percent of overpasses need repair, with a margin of error of 11 percent. To increase the precision of your estimate, you need to sample more. If you go to 200 overpasses (assuming you obtain the same 20 percent figure), your margin of error reduces to about six percent.

this conventional explanation is wrong: Lusinchi, D. (2012). “President” Landon and the 1936 Literary Digest poll: were automobile and telephone owners to blame? Social Science History, 36(1), 23–54.

An investigation uncovered serious flaws: Clement, S. (2013, June 4). Gallup explains what went wrong in 2012. Washington Post. https://www.washingtonpost.com/news/the-fix/wp/2013/06/04/gallup-explains-what-went-wrong-in-2012/.

http://www.gallup.com/poll/162887/gallup-2012-presidential-election-polling-review.aspx.

trying to figure out what proportion of jelly beans: Taken from http://www.ropercenter.uconn.edu/support/polling-fundamentals-total-survey-error/.

what magazines people read: Elaborated from an example in Huff, op. cit., p. 16.

Gleason scoring: This definition taken verbatim from http://www.cancer.gov/publications/dictionaries/cancer-terms?cdrid=45696. Accessed March 20, 2016.

they had made an error in measurement: Jordans, F. (2012, Feb. 23). CERN researchers find flaw in faster-than-light measurement. Christian Science Monitor. http://www.csmonitor.com/Science/2012/0223/CERN-researchers-find-flaw-in-faster-than-light-measurement.

1960 U.S. Census study recorded: This is from De Veaux, R. D., & Hand, D. J. (2005). How to lie with bad data. Statistical Science, 20(3), 231–238, p. 232. They cite Kruskal, W. (1981). Statistics in society: problems unsolved and unformulated. Journal of the American Statistical Association, 76(375), 505–515, and Coale, A. J., & Stephan, F. F. (1962). The case of the Indians and the teen-age widows. Journal of the American Statistical Association, 57, 338–347.

claimed measurement error as part of their defense: Kryk, J. Patriots strike back with compelling explanations to refute deflate-gate chargers. Ottowa Sun, May 15, 2015. http://www.ottawasun.com/2015/05/14/patriots-strike-back-with-compelling-explanations-to-refute-deflate-gate-chargers.

statistic you encounter may not have defined homelessness: This example from Spirer, H., Spirer, L., & Jaffe, A. J. (1998). Misused Statistics, 2nd ed., revised and expanded. New York: Marcel Dekker, p. 16.

Imagine that you’ve been hired by a political candidate: This example based on one in Huff, op. cit., p. 80.

A newspaper reports the proportion of suicides: From Best (2005), op. cit.

I’m not going to wear my seat belt because: This example comes from Best, J. (2012), and my childhood friend Kevin.

the idea of symmetry and equal likelihood: The principle of symmetry can be broadly construed to include instances where outcomes are not equally likely but still prescribed, such as a trick coin that is weighted to come up heads two-thirds of the time, or a roulette wheel in which some of the troughs are wider than others.

If we run the experiment on a large number of people: We could also conduct the experiment with a small number of people many times, in which case we would expect to obtain different numbers. In this case, the true probability of the drug working is going to be somewhere close to the average (the mean) of the numbers obtained in all the experiments, but it’s an axiom of statistics that larger samples lead to more accurate results.

Both classic and frequentist probabilities deal with: Classic probability can be thought of in two different ways: empirical and theoretical. If you’re going to toss a coin or draw cards from a shuffled deck, each time you do this is like a trial in an experiment that could go on indefinitely. In theory, you could get thousands of people to toss coins and pick cards for several years and tally up the results to obtain the proportion of time that different outcomes occur, such as “getting heads” or “getting heads three times in a row.” This is an empirically derived probability. If you believe the coin is fair (that is, there’s no manufacturing defect that causes it to come up on one side more than the other), you don’t need to do the experiment, because it should come up heads half of the time (probability = .5) in the long run, and we arrive at this theoretically, based on the understanding that there are two equally likely outcomes. We could run a similar experiment with cards and determine empirically and theoretically that the chances of drawing a heart are one in four (probability = .25) and that the chances of drawing the four of clubs is one in fifty-two (probability ≅ .02).

When a court witness testifies about the probability: Aitken, C. G. G., & Taroni, F. (2004). Statistics and the Evaluation of Evidence for Forensic Scientists, 2nd ed. Chicester, UK: John Wiley & Sons.

In Tversky and Kahneman’s experiments: Tversky, A., & Kahneman, D. (1974). Judgment under uncertainty: heuristics and biases. Science, 185(4157), 1124–1131.

A telltale piece of evidence that this is subjective: For further discussion, and more formal treatment, see Iversen, G. R. (1984). Bayesian Statistical Inference. Thousand Oaks, CA: Sage, and references cited therein.

the case of Sally Clark: I thank my student Alexandra Ghelerter for this example. See also Nobles, R., & Schiff, D. (2007). Misleading statistics within criminal trials. Medicine, Science and the Law, 47(1), 7–10.

relative incidence of pneumonia: http://www.nytimes.com/health/guides/disease/pneumonia/prognosis.html.

Bayes’s rule to calculate a conditional probability: Bayes’s rule is:

P(A | B) =	P(B | A) x P(A)
	P(B)

The probability that a woman has breast cancer: This paragraph, and this discussion, quotes nearly verbatim from Krämer, W., & Gigerenzer, G. (2005). How to confuse with statistics or: the use and misuse of conditional probabilities. Statistical Science, 20(3), 223–230.

To make the numbers work out easily: How do you know what number to choose? Sometimes it takes trial and error. But it’s also possible to figure it out. Because the probability is .8 percent, or eight people per thousand, if you chose to build a table for 1,000 women you’d end up with eight in one of the squares, and that’s okay, but later on we’re going to be multiplying that by 90 percent, which will give us a decimal. There’s nothing wrong with that, it’s just less convenient for most people to work with decimals. Increasing our population by an order of magnitude to 100 gives us all whole numbers, but then we’re looking at larger numbers than we need. It doesn’t really matter because all we’re looking for is probabilities and we’ll be dividing one number by another anyway for the result.

If you read that more automobile accidents occur at seven p.m.: Still confused? If there were eight times as many cars on the road at seven p.m. than at seven a.m., the raw number of accidents could be higher at seven p.m., but that does not necessarily mean that the proportion of accidents to cars is greater. And that is the relevant statistic to you: not how many accidents happen at seven p.m., but how many accidents occur per thousand cars on the road. This latter formulation quantifies your risk. This example is modified from one in Huff, op. cit., p. 78, and discussed by Krämer & Gigerenzer (2005).

90 percent of doctors treated the two: Cited in Spirer, Spirer, & Jaffe, op. cit., p. 197: Thompson, W. C., & Schumann, E. L. (1987). Interpretation of statistical evidence in criminal trials, Law and Human Behavior, 11(167).

One surgeon persuaded ninety women: From Spirer, Spirer, & Jaffe, op. cit., first reported in Hastie, R., & Dawes, R. M. (1988). Rational Choice in an Uncertain World. New York: Harcourt Brace Jovanovich.

The original report of the surgeon’s work appeared in McGee, G. (1979, Feb. 6). Breast surgery before cancer. Ann Arbor News, p. B1 (reprinted from the Bay City News).

As sociologist Joel Best says: Best, op. cit., p. 184.

PART TWO: EVALUATING WORDS

Steve Jobs delayed treatment for his pancreatic cancer: Swaine, J. (2011, Oct. 21). Steve Jobs “regretted trying to beat cancer with alternative medicine for so long.” http://www.telegraph.co.uk/technology/apple/8841347/Steve-Jobs-regretted-trying-to-beat-cancer-with-alternative-medicine-for-so-long.html.

an article in Forbes that claims: Rees, N. (2009, Aug. 13). Policing word abuse. Forbes. http://www.forbes.com/2009/08/12/nigel-rees-misquotes-opinions-rees.html.

Respectfully Quoted, a dictionary of quotations: Platt, S., ed. (1989). Respectfully Quoted. Washington, D.C.: Library of Congress. For sale by the Supt. of Docs., USGPO.

That book reports various formulations: Billings, J. (1874). Everybody’s Friend, or Josh Billing’s Encyclopedia and Proverbial Philosophy of Wit and Humor. Hartford, CT: American Publishing Company.

humans had twenty-four pairs of chromosomes instead of twenty-three: Gartler, S. M. (2006). The chromosome number in humans: a brief history. Nature Reviews Genetics, 7, 655–660. http://www.nature.com/scitable/con tent/The-chromosome-number-in-humans-a-brief-15575. Glass, B. (1990). Theophilus Shickel Painter. Washington, D.C.: National Academy of Sciences. http://www.nasonline.org/publications/biographical-memoirs/memoir-pdfs/painter-theophilus-shickel.pdf. Retrieved November 6, 2015.

If people in the arts and humanities have won a prize: Paul Simon, Stevie Wonder, and Joni Mitchell can be considered experts in songwriting. Although they do not hold university positions, university scholars have written books and articles about them, and Mr. Simon and Mr. Wonder were recognized by the president of the United States with Kennedy Center Honors, reserved for individuals who have made great contributions to performing arts. Ms. Mitchell received an honorary doctorate of music and won the Polaris Music Prize.

Some people, including Noam Chomsky, have argued: Chomsky, N. (2015, May 25). The New York Times is pure propaganda. Salon. http://www.salon.com/2015/05/25/noam_chomsky_the_new_york_times_is_pure_proganda_partner/.

Achbar, M., Symansky, A., & Wintonick, P. (Producers), and Achbar, M., & Wintonick, P. (Directors). (1992). Manufacturing Consent: Noam Chomsky and the Media (Motion picture). USA: BuyIndies.com Inc. and Zeitgeist Films. https://www.youtube.com/watch?v=BsiBl2CaDFg.

A 2011 fake tweet: Melendez, E. D. (2013, Feb. 1). Twitter stock market hoax draws attention of regulators. http://www.huffingtonpost.com/2013/02/01/twitter-stock-market-hoax_n_2601753.html; http://www.forbes.com/forbes/welcome/.

“The use of false rumors and news reports”: Farrell, M. (2015, July 14). Twitter shares hit by takeover hoax. Wall Street Journal. http://www.wsj.com/articles/twitter-shares-hit-by-takeover-hoax-1436918683.

Jonathan Capehart wrote a story: (2010, Sept. 7). Washington Post writer falls for fake congressman Twitter account. Huffington Post, updated Sept. 7, 2010. http://www.huffingtonpost.com/2010/09/07/washington-post-writer-fa_n_707132.html; http://voices.washingtonpost.com/postpartisan/2010/09/obama_deficits_and_the_ditch.html.

Who is behind it?: This is taken verbatim from The Organized Mind. Levitin, D. J. (2014). The Organized Mind. New York: Dutton.

2014 congressional race for Florida’s thirteenth district: Leary, A. (2014, Feb. 4). Misleading GOP website took donation meant for Alex Sink. Tampa Bay Times. http://www.tampabay.com/news/politics/stateroundup/misleading-gop-website-took-donation-meant-for-alex-sink/2164138.

A court case ruled that Degil: Pink, D. (2013). Deceiving domain names not allowed. Wickwire Holm. http://www.wickwireholm.com/Portals/0/newsletter/BLU%20Newsletter%20-%20January%202013%20-%20Deceiving %20Domain%20Names%20Not%20Allowed.pdf; Bonni, S. (2014, June 24). The tort of domain name passing off. Charity Law Bulletin 342, Carters Professional Corporation. http://www.carters.ca/pub/bulletin/charity/2014/chylb342.htm.

vendor operated the website GetCanadaDrugs.com: https://www.canada drugs.com/; https://www.getcanadadrugs.com/ (no longer available); Naud, M. (n.d.). Registered trade-mark canadadrugs.com found deceptively misdescriptive. ROBIC. http://www.robic.ca/admin/pdf/682/293.045E-MNA2007.pdf.

MartinLutherKing.org contains is a shameful assortment: The inflammatory quote from the website comes from the book by Taylor Branch, Pillar of Fire, but the author notes that he did not hear the tapes himself, he took them from three FBI agents who reported them to him.

Stormfront, a white-supremacy, neo-Nazi hate group.: Sources which identify Stormfront as the Internet’s “first hate site” include:

Levin, B. (2003). “Cyberhate: A legal and historical analysis of extremists’ use of computer networks in America,” in Perry, B., ed., Hate and Bias Crime: A Reader. New York: Routledge, p. 363.

Ryan, N. (2004). Into a World of Hate: A Journey Among the Extreme Right. New York: Routledge, p. 80.

Samuels, S. (1997). “Is the Holocaust unique?,” in Rosenbaum, Alan S., ed., Is the Holocaust Unique?: Perspectives on Comparative Genocide. Boulder, CO: Westview Press, p. 218.

Bolaffi, G.; et al., eds (2002). Dictionary of Race, Ethnicity and Culture. Thousand Oaks, CA: Sage Publications, p. 254.

Energy-drink company Red Bull paid: O’Reilly, L. (2014, Oct. 8). Red Bull will pay $10 to customers disappointed the drink didn’t actually give them “wings.” http://www.businessinsider.com/red-bull-settles-false-advertising-lawsuit-for-13-million-2014-10.

Target agreed to pay $3.9 million: Associated Press. (2015, Feb. 11). Target agrees to pay $3.9 million in false-advertising lawsuit. http://journal record.com/2015/02/11/target-agrees-to-pay-3-9-million-in-false-advertising-lawsuit-law/.

Kellogg’s paid $4 million to settle: Federal Trade Commission. (2009, April 20). Kellogg settles FTC charges that ads for Frosted Mini-Wheats were false [Press release]. https://www.ftc.gov/news-events/press-releases/2009/04/kellogg-settles-ftc-charges-ads-frosted-mini-wheats-were-false.

The Washington Post also runs a fact-checking site: https://www.washing tonpost.com/news/fact-checker/.

Politifact summarized its findings: Carroll, L. (2015, Nov. 22). Fact-checking Trump’s claim that thousands in New Jersey cheered when World Trade Center tumbled. http://www.politifact.com/truth-o-meter/statements/2015/nov/22/donald-trump/fact-checking-trumps-claim-thousands-new-jersey-ch/.

only one grandparent was born abroad: Sanders, K. (2015, April 16). In Iowa, Hillary Clinton claims “all my grandparents” came to the U.S. from foreign countries. http://www.politifact.com/truth-o-meter/statements/2015/apr/16/hillary-clinton/hillary-clinton-flubs-familys-immigration-history-/.

322,000,000: The population of the United States, as of this writing. http://www.census.gov/popclock/.

For coronary heart disease: American Heart Association (2015). AHA Statistical Update. Circulation, 131, p. 434–441. I thank McGill University Librarians Robin Canuel and Genevieve Gore for help in finding these statistics.

In 1968, Will and Ariel Durant wrote: Durant, W., & Durant, A. (1968). The Lessons of History. New York: Simon & Schuster.

The FBI announced in 2015: Federal Bureau of Investigation (2015, April 20). FBI testimony on microscopic hair analysis contained errors in at least 90 percent of cases in ongoing review [Press release]. https://www.fbi.gov/news/pressrel/press-releases/fbi-testimony-on-microscopic-hair-analysis-contained-errors-in-at-least-90-percent-of-cases-in-ongoing-review.

Without these pieces of information: Aitken, C. G. G., & Taroni, F. (2004). Statistics and the Evaluation of Evidence for Forensic Scientists, 2nd ed. Chicester, UK: John Wiley & Sons, p. 95, citing Friedman, R. D. (1996). Assessing Evidence. Michigan Law Review, 94(6), 1810–1838.

In one case in the U.K.: R v. Dennis John Adams, (1996) 2 Cr App R, 467;

And Aitken, C. (2003). Statistical techniques and their role in evidence interpretation. In Payne-James, J., Busuttil, A., & Smock, W., eds., Forensic Medicine: Clinical and Pathological Aspects. Cambridge, UK: Cambridge University Press.

the New York Times described a mysterious formation: Blumenthal, R. (2015, Nov. 3). Built by the ancients, seen from space. New York Times, p. D2.

How much more productive and creative might she have been: I thank Stephen Kosslyn for sharing a version of this example with me.

Two twins were separated at birth: Grimes, W. (2015, Nov. 13). Jack Yufe, a Jew whose twin was a nazi, dies at 82. New York Times, p. B8.

They were reunited twenty-one years later: Much of this is taken verbatim from Grimes (2015), op. cit.

A statistician or behavioral geneticist would say: Dr. Jeffrey Mogil, personal communication.

if you ask a hundred people in a room: The formula is 1 – (1 – 1/25)100.

Paul McCartney and Dick Clark: I thank Ron Mann for this observation.

Larger samples more accurately reflect: Note that in a large sample, you are more likely to find an anomalous (outlier) observation than in a small sample, but when looking at the mean, the mean of the large sample is far more likely to reflect the true state of the world (because there are so many more observations that can swamp the anomalous one).

if the study was on the incidence of preterm births: Krämer, W., & Gigerenzer, G. (2005). How to confuse with statistics or: the use and misuse of conditional probabilities. Statistical Science, 20(3), 223–230. See also Centers for Disease Control and Prevention. Preterm birth. http://www.cdc.gov/reproductivehealth/maternalinfanthealth/pretermbirth.htm.

Consider a street game in which a hat: Krämer, W., & Gigerenzer, G. (2005). Technically, they note, this is an incorrect enumeration of simple events in a Laplacian experiment in the subpopulation composed of the remaining possibilities.

similar mistakes were made by mathematical philosopher Gottfried Wilhelm Leibniz: Ibid.

Counterknowledge, a term coined by: Thompson, D. (2008). Counterknowledge: How We Surrendered to Conspiracy Theories, Quack Medicine, Bogus Science, and Fake History. New York: W. W. Norton, p. 1.

Damian Thompson tells the story: Thompson, D. (2008), op. cit.

Shot on a consumer-grade camera: Trask, R. B. (1996). Photographic Memory: The Kennedy Assassination, November 22, 1963. Dallas: Sixth Floor Museum, p. 5.

A handful of unexplained anomalies: Thanks to Michael Shermer for this.

The difference between a false theory: This is a direct quote from Thompson, D. (2008), op. cit.

As Damian Thompson notes: Thompson, D. (2008), op. cit., p. 17. The previous two sentences are from pp. 16–17.

die each year of stomach cancer: National Cancer Institute. SEER stat fact sheets: stomach cancer. http://seer.cancer.gov/statfacts/html/stomach.html.

than of unintentional drowning: Centers for Disease Control and Prevention. Unintentional drowning: get the facts. http://www.cdc.gov/Homeand RecreationalSafety/Water-Safety/waterinjuries-factsheet.html.

A front-page headline in the Times (U.K.): Smyth, C. (2015, Feb. 4). “Half of all Britons will get cancer during their lifetime.” Times. www.thetimes.co.uk/tto/health/news/article4343681.ece.

Cancer Research UK (CRUK) reports that: Boseley, S. (2015, Feb. 3). Half of people in Britain born after 1960 will get cancer, study shows. Guardian.

Heart disease is better controlled: Griffiths, C., & Brock, A. (2003). Twentieth century mortality trends in England and Wales. Health Statistics Quarterly, 18(2), 5–17.

This is based on reports by a variety: http://www.nrdc.org/water/drinking/qbw.asp; http://www.mayoclinic.org/healthy-lifestyle/nutrition-and-healthy-eating/expert-answers/tap-vs-bottled-water/faq-20058017; http://www.consumerreports.org/cro/news/2009/07/is-tap-water-safer-than-bottled/index.htm; http://news.nationalgeographic.com/news/2010/03/100310/why-tap-water-is-better/; http://abcnews.go.com/Business/study-bottled-water-safer-tap-water/story?id=87558; http://www.telegraph.co.uk/news/health/news/9775158/Bottled-water-not-as-safe-as-tap-variety.html.

In New York City; Montreal; Flint, Michigan; and many other older cities: Stockton, N. (2016, Jan. 29). Here’s how hard it will be to unpoison Flint’s water. Wired. http://www.wired.com/2016/01/heres-how-hard-it-will-be-to-unpoison-flints-water/.

PART THREE: EVALUATING THE WORLD

Nature permits us to calculate only probabilities: Feynman, R. P. (1985). QED: The Strange Theory of Light and Matter. Princeton, NJ: Princeton University Press.

A case of fraud occurred in 2015: Reardon, S. (2015, July 1). US vaccine researcher sentenced to prison for fraud. Nature, 523, p. 138.

controversy about whether the measles, mumps, and rubella MMR vaccine causes autism: Wakefield, A. J., et al. (1998, Feb. 28). RETRACTED: Ileal-lymphoid-nodular hyperplasia, non-specific colitis, and pervasive developmental disorder in children. Lancet, 351(9103), 637–641. http://www.thelancet.com/journals/lancet/article/PIIS0140-6736(97)11096-0/abstract.

Burns, J. F. (2010, May 25). British medical council bars doctor who linked vaccine with autism. New York Times, p. A4. http://www.nytimes.com/2010/05/25/health/policy/25autism.html.

Associated Press (2011, Jan. 6). Study linking vaccine to autism is called fraud. New York Times. http://query.nytimes.com/gst/fullpage.html?res=9C02E7DC1E3BF935A35752C0A9679D8B63.

Rao, T. S., & Andrade, C. (2011). The MMR vaccine and autism: sensation, refutation, retraction, and fraud. Indian Journal of Psychiatry, 53(2), 95–96.

For example, Holmes concludes that: From Thompson, S. (2010). The blind banker. Sherlock (TV series, first aired October 31, 2010).

The germ theory of disease: I first learned about this story from Hempel, C. (1966). Philosophy of Natural Science. Englewood Cliffs, NJ: Prentice-Hall.

To fill out the rest of the table: I’m using the 168-hour week (7 days × 24 hours a day) to account for thoughts you might have while dreaming, and people who might call and wake you from a sound sleep. Of course, one could subtract out eight hours of sleep per night (or whatever) and then use only the 112 hours of wakefulness to come up with a different probability, but it doesn’t change the conclusion.

less safe mode of travel: In retrospect this switch was foolish, at the time, but it may have been the rational thing to do. Four hijacked, suicide planes at once was unprecedented in aviation history. When confronted with a big change in the world, often the best thing to do is to think Bayesian: update your understanding, stop relying on the old statistics, and seek alternatives.

conclude that air travel: Based on Huff, op. cit., p. 79.

There were not nearly as many flights in 1960: See, for example, Iolan, C., Patterson, T., & Johnson, A. (2014, July 28). Is 2014 the deadliest year for flights? Not even close. CNN. http://www.cnn.com/interactive/2014/07/travel/aviation-data/; and Evershed, N. (2015, March 24). Aircraft accident rates at historic low despite high-profile plane crashes. Guardian. http://www.theguardian.com/world/datablog/2014/dec/29/aircraft-accident-rates-at-historic-low-despite-high-profile-plane-crashes.

An FBI page reports that: http://www.fbi.gov/about-us/cjis/ucr/crime-in-the-u.s/2011/crime-in-the-u.s.-2011/clearances.

All home robberies in a neighborhood: Image from http://contactglenda.com/wp-content/uploads/2011/08/robbers-decamp.png.

In a famous psychology experiment: Nisbett, R. E., & Valins, S. (1972). Perceiving the causes of one’s own behavior. In Kanouse, D. E., et al., eds. Attribution: perceiving the causes of behavior. Morristown, NJ: General Learning Press, pp. 63–78.

And, Valins, S. (2007). Persistent effects of information about internal reactions: ineffectiveness of debriefing. In London, H., & Nisbett, R. E., eds. Thought and Feeling: the cognitive alteration of feeling states. Chicago, IL: Aldine Transaction.

Between 1990 and 2010, the number of children diagnosed with autism spectrum disorders (ASD) rose sixfold: What is causing the increase in autism prevalence. Autism Speaks Official Blog, Oct. 22, 2010. http://blog.autismspeaks.org/2010/10/22/got-questions-answers-to-your-questions-from-the-autism-speaks%E2%80%99-science-staff-2/.

The majority of the rise: Ibid.

the Internet to guide your thinking on why autism: Suresh, A. (2015, Oct. 13). Autism increase mystery solved: no, it’s not vaccines, GMOs, glyphosate—or organic foods. Genetic Literacy Project. http://www.geneticliteracyproject.org/2015/10/13/autism-increase-mystery-solved-no-its-not-vaccines-gmos-glyphosate-or-organic-foods/.

She also couches her argument: Kase, A. (2015, May 11). MIT scientist uncovers link between glyphosate, GMOs and the autism epidemic. Reset.me. http://reset.me/story/mit-scientist-uncovers-link-between-glyphosate-gmos-and-the-autism-epidemic/.

no evidence that thimerosal was linked to autism: Honda, H., Shimizu, Y., & Rutter, M. (2005). No effect of MMR withdrawal on the incidence of autism: a total population study. Journal of Child Psychology and Psychiatry, 46(6), 572–579. http://1796kotok.com/pdfs/MMR_withdrawal.pdf, and many other sources.

Reardon, S. (2015). US vaccine researcher sentenced to prison for fraud. Nature, 523(7559), p. 138.

as we know, there are known knowns: Defense.gov News Transcript: DoD News Briefing—Secretary Rumsfeld and Gen. Myers, United States Department of Defense (defense.gov).

We can clarify Secretary Rumsfeld’s four possibilities with a fourfold table: I thank Morris Olitsky for this.

One of the cornerstone principles of forensic science: Inman, K., & Rudin, N. (2002). The origin of evidence. Forensic Science International, 126(1), 11–16.

Inman, K., & Rudin, N. (2000). Principles and Practice of Criminalistics: the profession of forensic science. Boca Raton, FL: CRC Press.

Suppose a criminal breaks into the stables: I’m basing this section on the discussion found in Aitken, C. G. G., & Taroni, F. (2004). Statistics and the Evaluation of Evidence for Forensic Scientists, 2nd ed. Chicester, UK: John Wiley & Sons, pp. 1–2, and using their setup and terminology.

take literally the assumption in the American legal system: Aitken, C. G. G., & Taroni, F. (2004), op. cit.

the prosecutor’s fallacy: Thompson, W. C.; Shumann, E. L. (1987). Interpretation of statistical evidence in criminal trials: the prosecutor’s fallacy and the defense attorney’s fallacy. Law and Human Behavior 2(3), 167–187.

The quality of the photographs is high: Hasselblad.com. https://www.hq.nasa.gov/alsj/a11/a11-hass.html; http://www.wired.com/2013/07/apollo-hasselblad/.

It’s been claimed that the chances of life forming on Earth: Estimates include 1 x 10390. http://evolutionfaq.com/articles/probability-life. See also Dreamer, D. (2009, April 30). Calculating the odds that life could begin by chance. Science 2.0. http://www.science20.com/stars_planets_life/calculating_odds_life_could_begin_chance.

In a TED talk with more than 10 million views: https://www.ted.com/talks/david_blaine_how_i_held_my_breath_for_17_min?language=en.

there are more than 5,000 TED-branded events: Bruno Guissani, Curator of TEDGlobal Conference, personal communication, September 28, 2015.

Fox television reported his ice-block demonstration: https://www.you tube.com/watch?v=U6Em2OhvEJY.

Out of a sense of ethics: Glenn Falkenstein, personal communication, October 25, 2007.

There was even a peer-reviewed paper: Korbonits, M., Blaine, D., Elia, M., & Powell-Tuck, J. (2005). Refeeding David Blaine—studies after a 44-day fast. New England Journal of Medicine, 353(21), 2306–2307.

The current editor of the journal searched: J. Drazen, MD, email communication, December 20, 2015.

The lead author on the article told me in an email: M. Korbonits, MD, email communication, December 25, 2015.

a physician did monitor Blaine throughout the fast: Jackson, J. M., et al. (2006). Macro- and micronutrient losses and nutritional status resulting from 44 days of total fasting in a non-obese man. Nutrition, 22(9), 889–897.

Blaine’s record was broken in 2012: http://www.guinnessworldrecords.com/world-records/24135-longest-time-breath-held-voluntarily-male; Grenoble, R. (2012, Nov. 16). Breath-holding world record: Stig Severinsen stays under water for 22 minutes (Video), Huffington Post. http://www.huffing tonpost.com/2012/11/16/breath-world-record-stig-severinsen_n_2144734.html.

An article in the Dallas Observer: Liner, E. (2012, Jan. 13). Want to know how David Blaine does that stuff? (Don’t hold your breath). http://www.dallasobserver.com/arts/want-to-know-how-david-blaine-does-that-stuff-dont-hold-your-breath-7083351.

preparation for the breath holding for an article in the New York Times: Tierney, J. (2008, April 22). This time, he’ll be left breathless. New York Times, p. F1.

the Oprah appearance in his blog: Tierney, J. (2008). David Blaine sets breath-holding record. http://tierneylab.blogs.nytimes.com/2008/04/30/david-blaine-sets-breath-holding-record.

Tierney writes, “I was there”: John Tierney, email correspondence, January 13 and 18, 2016.

Eventually all of the elements between 1 and 118: Netburn, D. (2016, Jan. 4). It’s official: four super-heavy elements to be added to the periodic table. http://www.latimes.com/science/sciencenow/la-sci-sn-new-elements-20160104-story.html.

Here’s Professor Harrison Prosper, describing this plot: Prosper, H. B. (2012, July 10). International Society for Bayesian Analysis. http://bayesian.org/forums/news/3648.

Louis Lyons explains “The Higgs”: Lyons, L. (2012, July 11). http://bayesian.org/forums/news/3648.

Although CERN officials announced in 2012: In articles on the Higgs boson, you may encounter reference to the 5-sigma standard of proof. Five-sigma refers to the level of probability that the scientists agreed upon before conducting the experiments—the chance of their misinterpreting the experiments had to have a confidence interval within five standard deviations (5-sigma), or 0.0000005 (recall earlier we talked about 95 and 99 percent confidence intervals—this is a confidence interval of 99.99995 percent). See http://blogs.scientificamerican.com/observations/five-sigmawhats-that/.

Prosper says, “Given that the search”: Prosper, H. B. (2012, July 10). http://bayesian.org/forums/news/3648.

Physicist Mads Toudal Frandsen adds: (2014, Nov. 7). Maybe it wasn’t the Higgs particle after all. Phys.org. http://phys.org/news/2014-11-wasnt-higgs-particle.html.

Joseph Lykken, a physicist and director of the Fermi National Accelerator Laboratory: http://phys.org/news/2014-11-wasnt-higgs-particle.html.

as Wired writer Signe Brewster says: http://www.wired.com/2015/11/physicists-are-desperate-to-be-wrong-about-the-higgs-boson/.

physicist Nima Arkani-Hamed told the New York Times: Overbye, D. (2015, Dec. 16). Physicists in Europe find tantalizing hints of a mysterious new particle. New York Times, p. A16.

CONCLUSION: DISCOVERING YOUR OWN

A lot of things that should be scientific: These ideas and their phrasing come from: Frum, D. (2015). Talk delivered at the Colleges Ontario Higher Education Summit, November 16, 2015, Toronto, ON.

APPENDIX: APPLICATION OF BAYES’S RULE

To compute Bayes’s rule: Iversen, G. R. (1984). Bayesian Statistical Inference. Quantitative Applications in the Social Sciences, vol. 43. Thousand Oaks, CA: Sage.