Chapter 2

Probability

In this chapter, you’ll learn the rules of probability and apply them to games of chance, jury selection, surveys, and blood types. You’ll use R to generate simulated random data and learn how to create your own R functions.

2.1 PROBABILITY

Because of the variation that is inherent in the processes we study, we are forced to speak in probabilistic terms rather than absolutes. We talk about the probability that a sixth-grader is exactly 150 cm tall, or, more often, the probability that his height will lie between two values, such as 150 and 155 cm. The events we study may happen a large proportion of the time, or “almost always,” but seldom “always” or “never.”

Rather arbitrarily, and some time ago, it was decided that probabilities would be assigned a value between 0 and 1, that events that were certain to occur would be assigned probability 1, and that events that would “never” occur would be given probability zero. This makes sense if we interpret probabilities in terms of frequencies; for example, that the probability that a coin turns up heads is the ratio of the number of heads in a very large number of coin tosses to the total number of coin tosses.

When talking about a set of equally likely events, such as the probability that a fair coin will come up heads, or an unweighted die will display a “6,” this limitation makes a great deal of sense. A coin has two sides; we say the probability it comes up heads is a half and the probability of tails is a half also. 1/2 + 1/2 = 1, the probability that a coin comes up something.* Similarly, the probability that a six-sided die displays a “6” is 1/6. The probability it does not display a 6 is 1 − 1/6 = 5/6.

For every dollar you bet, Roulette wheels pay off $36 if you win. This certainly seems fair, until you notice that not only does the wheel have slots for the numbers 1 through 36, but there is a slot for 0, and sometimes for double 0, and for triple 000 as well. The real probabilities of winning and losing in this latter case are, respectively, 1 chance in 39 and 38/39. In the long run, you lose one dollar 38 times as often as you win $36. Even when you win, the casino pockets your dollar, so that in the long run, the casino pockets $3 for every $39 that is bet. (And from whose pockets does that money come?)

Ah, but you have a clever strategy called a martingale. Every time you lose, you simply double your bet. So if you lose a dollar the first time, you lose two dollars the next. Hmm. Since the casino always has more money than you do, you still end up broke. Tell me again why this is a clever strategy.

Exercise 2.1: List the possible ways in which the following can occur:

a. A specific person, call him Bill, is born on a Wednesday.
b. Bill and Alice are born on the same day of the week.
c. Bill and Alice are born on different days of the week.
d. Bill and Alice play a round of a game called “paper, scissor, stone” and simultaneously display either an open hand, two fingers, or a closed fist.

Exercise 2.2: Match the probabilities with their descriptions. A description may match more than one probability.

c02t2n7za

To determine whether a gambling strategy or a statistic is optimal, we need to know a few of the laws of probability. These laws show us how to determine the probabilities of combinations of events. For example, if the probability that an event A will occur is P{A}, then the probability that A won’t occur P{notA} = 1 − P{A}. This makes sense since either the event A occurs or it doesn’t, and thus P{A} + P{notA} = 1.

We’ll also be concerned with the probability that both A and B occur, P{A and B}, or with the probability that either A occurs or B occurs or both do, P{A or B}. If two events A and B are mutually exclusive, that is, if when one occurs, the other cannot possibly occur, then the probability that A or B will occur, P{A or B} is the sum of their separate probabilities. (Quick, what is the probability that both A and B occur.)* The probability that a six-sided die will show an odd number is thus 3/6 or 1/2. The probability that a six-sided die will not show an even number is equal to the probability that a six-sided die will show an odd number.

Just because there are exactly two possibilities does not mean that they are equally likely. Otherwise, the United States would never be able to elect a President. Later in this chapter, we’ll consider the binomial distribution, which results when we repeat a sequence of trials, each of which can result in exactly one of two possibilities.

2.1.1 Events and Outcomes

An outcome is something we can observe. For example, “the coin lands heads” or “an odd number appears on the die.” Outcomes are made up of events that may or may not be completely observable. The referee tosses the coin into the air; it flips over three times before he catches it and places it face upward on his opposite wrist. “Heads,” and Manchester United gets the call. But the coin might also have come up heads had the coin been tossed higher in the air so that it spun three and a half or four times before being caught. A literal infinity of events makes up the single observed outcome, “Heads.”

The outcome “an odd number appears on the six-sided die,” is composed of three outcomes, 1, 3, and 5, each of which can be the result of any of an infinity of events. By definition, events are mutually exclusive. Outcomes may or may not be mutually exclusive, depending on how we aggregate events.

2.1.2 Venn Diagrams

An excellent way to gain insight into the distinction between events and outcomes and the laws of probability is via the Venn diagram.* Figure 2.1 pictures two overlapping outcomes, A and B. For example, the set A might consist of all those who respond to a survey that they are non-smokers, while B corresponds to the set of all respondents that have lung cancer.

Figure 2.1 Venn Diagram depicting two overlapping outcomes.

c02f001

Every point in the figure corresponds to an event. The events within the circle A all lead to the outcome A. Note that many of the events or points in the diagram lie outside both circles. These events correspond to the outcome, “neither A nor B” or, in our example, “an individual who does smoke and does not have lung cancer.”

The circles overlap; thus outcomes A and B are not mutually exclusive. Indeed, any point in the region of overlap between the two, marked C, leads to the outcome “A and B.” What can we say about individuals who lie in region C?

Exercise 2.3: Construct a Venn diagram corresponding to the possible outcomes of throwing a six-sided die. (I find it easier to use squares than circles to represent the outcomes, but the choice is up to you.) Does every event belong to one of the outcomes? Can an event belong to more than of these outcomes? Now, shade the area that contains the outcome, “the number face up on the die is odd.” Use a different shading to outline the outcome, “the number on the die is greater than 3.”

Exercise 2.4: Are the two outcomes “the number face up on the die is odd” and “the number on the die is greater than 3” mutually exclusive?

You’ll find many excellent Venn diagrams illustrating probability concepts at http://stat-www.berkeley.edu/users/stark/Java/Html/Venn.htm


In the Long Run: Some Misconceptions
When events occur as a result of chance alone, anything can happen and usually will. You roll craps seven times in a row, or you flip a coin 10 times and 10 times it comes up heads. Both these events are unlikely, but they are not impossible. Before reading the balance of this section, test yourself by seeing if you can answer the following:
You’ve been studying a certain roulette wheel that is divided into 38 sections for over 4 hours now and not once during those 4 hours of continuous play has the ball fallen into the number 6 slot. Which of the following do you feel is more likely?
1. Number 6 is bound to come up soon.
2. The wheel is fixed so that number 6 will never come up.
3. The odds are exactly what they’ve always been and in the next four hours number 6 will probably come up about 1/38th of the time.
If you answered (2) or (3) you’re on the right track. If you answered (1), think about the following equivalent question:
You’ve been studying a series of patients treated with a new experimental drug all of whom died in excruciating agony despite the treatment. Do you conclude the drug is bound to cure somebody sooner or later and take it yourself when you come down with the symptoms? Or do you decide to abandon this drug and look for an alternative?

2.2 BINOMIAL TRIALS

Many of our observations take a yes/no or dichotomous form: “My headache did/didn’t get better.” “Chicago beat/was beaten by Los Angeles.” “The respondent said he would/wouldn’t vote for Ron Paul.” The simplest example of a binomial trial is that of a coin flip: heads I win, tails you lose.

If the coin is fair, that is, if the only difference between the two mutually exclusive outcomes lies in their names, then the probability of throwing a head is 1/2, and the probability of throwing a tail is also 1/2. (That’s what I like about my bet, either way I win.)

By definition, the probability that something will happen is 1, the probability that nothing will occur is 0. All other probabilities are somewhere in between.*

What about the probability of throwing heads twice in a row? Ten times in a row? If the coin is fair and the throws independent of one another, the answers are easy: 1/4th and 1/1024th or (1/2)10.

These answers are based on our belief that when the only differences among several possible mutually exclusive outcomes are their labels, “heads” and “tails,” for example, the various outcomes will be equally likely. If we flip two fair coins or one fair coin twice in a row, there are four possible outcomes HH, HT, TH, and TT. Each outcome has equal probability of occurring. The probability of observing the one outcome in which we are interested is 1 in 4 or 1/4th. Flip the coin 10 times and there are 210 or a thousand possible outcomes; one such outcome might be described as HTTTTTTTTH.

Unscrupulous gamblers have weighted coins so that heads comes up more often than tails. In such a case, there is a real difference between the two sides of the coin and the probabilities will be different from those described above. Suppose as a result of weighting the coin, the probability of getting a head is now p, where 0 ≤ p ≤ 1, and the complementary probability of getting a tail (or not getting a head) is 1 − p, because p + (1 − p) = 1. Again, we ask the question, what is the probability of getting two heads in a row? The answer is pp or p2. Here is why:

To get two heads in a row, we must throw a head on the first toss, which we expect to do in a proportion p of attempts. Of this proportion, only a further fraction p of two successive tosses also end with a head, that is, only pp trials result in HH. Similarly, the probability of throwing ten heads in a row is p10.

By the same line of reasoning, we can show the probability of throwing nine heads in a row followed by a tail when we use the same weighted coin each time is p9(1 − p). What is the probability of throwing 9 heads in 10 trials? Is it also p9(1 − p)? No, for the outcome “nine heads out of ten” includes the case where the first trial is a tail and all the rest are heads, the second trial is a tail and all the rest are heads, the third trial is, and so forth. Ten different ways in all. These different ways are mutually exclusive, that is, if one of these events occurs, the others are excluded. The probability of the overall event is the sum of the individual probabilities or 10 p9(1 − p).

Exercise 2.5: What is the probability that if you flipped a fair coin you would get heads five times in a row?

Exercise 2.6: The strength of support for our candidate seems to depend on whether we are interviewing men or women. Fifty percent of male voters support our candidate, but only 30% of women. What percent of women favor some other candidate? If we select a woman and a man at random and ask which candidate they support, in what percentage of cases do you think both will say they support our candidate?

Exercise 2.7: Would your answer to the previous question be the same if the man and the woman were coworkers? family members? fellow Republicans?

Exercise 2.8: Which approach do you think would be preferable to use in customer-satisfaction survey? To ask customers if they were or were not satisfied? Or to ask them to specify their degree of satisfaction on a 5-point scale? Why?

2.2.1 Permutations and Rearrangements

What is the probability of throwing exactly five heads in 10 tosses of a coin? The answer to this last question requires we understand something about permutations and combinations or rearrangements, a concept that will be extremely important in succeeding chapters.

Suppose we have three horses in a race. Call them A, B, and C. A could come in first, B could come in second, and C would be last. ABC is one possible outcome or permutation. But so are ACB, BAC, BCA, CAB, CBA; that is, there are six possibilities or permutations in all. Now suppose we have a nine-horse race. To determine the total number of possible orders of finish, we could enumerate them all one by one or we could use the following trick: We choose a winner (nine possibilities); we choose a second place finisher (eight remaining possibilities), and so forth until all positions are assigned. A total of 9! = 9 × 8 × 7 × 6 × 5 × 4 × 3 × 2 × 1 possibilities in all. Had there been N horses in the race, there would have been N! possibilities. N! is read “N factorial.”

Using R

> factorial(N)

yields the value of N! for any integer N.

Normally in a horse race, all our attention is focused on the first three finishers. How many possibilities are there? Using the same reasoning, it is easy to see there are 9 × 8 × 7 possibilities or 9!/6!. Had there been N horses in the race, there would have been N!/(N–3)! possibilities.

Suppose we ask a slightly different question: In how many different ways can we select three horses from nine entries without regard to order, (i.e., we don’t care which comes first, which second, or which third). This would be important if we had bet on a particular horse to show, that is, to be one of the first three finishers in a race.

In the previous example, we distinguished between first, second, and third place finishers; now, we’re saying the order of finish doesn’t make any difference. We already know there are 3! = 3 × 2 × 1 = 6 different permutations of the three horses that finish in the first three places. So we take our answer to the preceding question 9!/6! and divide this answer in turn by 3!. We write the result as

c02ue001

which is usually read as 9 choose 3. Note that

c02ue002

In how many different ways can we assign nine cell cultures to two unequal experimental groups, one with three cultures and one with six? This would be the case if we had nine labels and three of the labels read “vitamin E” while six read “controls.” If we could distinguish the individual labels, we could assign them in 9! different ways. But the order they are assigned within each of the experimental groups, 3! ways in the first instance and 6! in the other, won’t affect the results. Thus, there are only 9!/6!3! or

c02ue003

We can generalize this result to show the number of distinguishable ways N items can be assigned to two groups, one of k items, and one of N – k is

c02ue004

Using R, we would write

> choose(N,k).

What if we were to divide these same nine cultures among three equal-sized experimental groups? Then we would have 9!/3!3!3! distinguishable ways or rearrangements, written as

c02ue005

Exercise 2.9: What is the value of 4!

Exercise 2.10: In how many different ways can we divide eight subjects into two equal-sized groups?

*2.2.2 Programming Your Own Functions in R

To tell whether our answer to question 2.10 is correct, we can program the factorial function in R.

c02uf001 fact = function (num) {
+ prod(1:num)
+ }

Here, “1:num” yields a sequence of numbers from 1 to num, and prod() gives the product of these numbers.

To check your answer to the exercise, just type fact(4) after the >. Note that this function only makes sense for positive integers, 1, 2, 3, and so forth. To forestall errors, here is an improved version of the program, in which we print solution messages whenever an incorrect entry is made:

c02uf002 fact = function (num) {
    if(num!=trunc(num))return(print(″argument must be whole number″))
    if(num<0) return(print (″argument must be nonnegative integer″))
    else if(n= =0)return(1)
    else return(prod(1:num))
    }

Note that != means not equal to, while = = means equivalent to. Warning: writing if (n = 0) by mistake would lead to setting n equal to zero. Always use the = = in an if statement.

In general, to define a function in R, we use the form

function_name = function (param1, param2, . . .){
  . . .
  return (answer)
  }

If we want the function to do different things depending on satisfying a certain condition, we write

c02uf003 if (condition) statement1 else statement2.

Possible conditions include X > 3 and name = = “Mike.” Note that in R, the symbol = = means “is equivalent to,” while the symbol = means “assign the value of the expression on the right of the equals sign to the variable on the left.”

*Exercise 2.11: Write an R function that will compute the number of ways we can choose m from n things. Be sure to include solution messages to avoid problems.

2.2.3 Back to the Binomial

We used horses in this last example, but the same reasoning can be applied to coins or survivors in a clinical trial.* What is the probability of five heads in 10 tosses? What is the probability that five of 10 breast cancer patients will still be alive after 6 months?

We answer these questions in two stages. First, what is the number of different ways we can get five heads in 10 tosses? We could have thrown HHHHHTTTTT or HHHHTHTTTT, or some other combination of five heads and five tails for a total of 10 choose 5 or 10!/(5!5!) ways. The probability the first of these events occurs—five heads followed by five tails—is (1/2)10. Combining these results yields

c02ue006

We can generalize the preceding to an arbitrary probability of success p, 0 ≤ p ≤ 1. The probability of failure is 1 − p. The probability of k successes in n trials is given by the binomial formula

(2.1) c02e001

Exercise 2.12: What is the probability of getting at least one head in six flips of a fair coin?

2.2.4 The Problem Jury

At issue in Ballew v. Georgia brought before the Supreme Court in 1978 was whether the all-white jury in Ballew’s trial represented a denial of Ballew’s rights. In the 1960s and 1970s, U.S. courts held uniformly that the use of race, gender, religion, or political affiliation to bar citizens from jury service would not be tolerated. In one case in 1963 in which I assisted the defense on appeal, we were able to show that only one black had served on some 163 consecutive jury panels. In this case, we were objecting—successfully—to the methods used to select the jury. In Ballew, the defendant was not objecting to the methods but to the composition of the specific jury that had judged him at trial.

In the district in which Ballew was tried, blacks comprised 10% of the population, but Ballew’s jury was entirely white. Justice Blackmun wanted to know what the probability was that a jury of 12 persons selected from such a population in accordance with the law would fail to include members of the minority.

If the population in question is large enough, say a hundred thousand or so, we can assume that the probability of selecting a nonminority juryperson is a constant 90 out of 100. The probability of selecting two nonminority persons in a row according to the product rule for independent events is 0.9 × 0.9 or 0.81. Repeating this calculation 10 more times, once for each of the remaining 10 jurypersons, we get a probability of 0.9 × 0.9 × … …  × 0.9 = 0.28243 or 28%.

Not incidentally, Justice Blackmun made exactly this same calculation and concluded that Ballew had not been denied his rights.

*2.3 CONDITIONAL PROBABILITY

Conditional probability is one of the most difficult of statistical concepts, not so much to understand as to accept in all its implications. Recall that mathematicians arbitrarily assign a probability of 1 to the result that something will happen—the coin will come up heads or tails, and 0 to the probability that nothing will occur. But real life is more restricted: a series of past events has preceded our present, and every future outcome is conditioned on this past. Consequently, we need a method whereby the probabilities of just the remaining possibilities sum to 1.

We define the conditional probability of an event A given another event B, written P(A|B), to be the ratio P(A and B)/P(B). To show how this would work, suppose we are playing Craps, a game in which we throw two six-sided die. Clearly, there are 6 × 6 = 36 possible outcomes. One (and only one) of these 36 outcomes is snake eyes, a 1 and a 1.

Now, suppose we throw one die at a time (a method that is absolutely forbidden in any real game of Craps, whether in the Bellagio or a back alley) and a 1 appears on the first die. The probability that we will now roll snake eyes, that is, that the second die will reveal a 1 also, is 1 out of 6 possibilities or (1/36)/(1/6) = 6/36 = 1/6.

The conditional probability of rolling a total of seven spots on the two dice is 1/6. And the conditional probability of the spots on the two die summing to 11, another winning combination, is 0. Yet before we rolled the two dice, the unconditional probability of rolling snake eyes was 1 out of 36 possibilities, and the probability of 11 spots on the two die was 2/36th (a 5 and a 6 or a 6 and a 5).

Now, suppose I walk into the next room where I have two decks of cards. One is an ordinary deck of 52 cards, half red and half black. The other is a trick deck in which all the spots on the cards are black. I throw a coin—I’m still in the next room so you don’t get to see the result of the coin toss—and if the coin comes up heads I stick the trick deck in my pocket, otherwise I take the normal deck. Now, I come back into the room and offer to show you a card chosen at random from the deck in my pocket. The card has black spots. Would you like to bet on whether or not I’m carrying the trick deck?

[STOP: Think about your answer before reading further.]

Common sense would seem to suggest that the odds are still only 50–50 that it’s the trick deck I’m carrying. You didn’t really learn anything from seeing a card that could have come from either deck. Or did you?

Let’s use our conditional probability relation to find out whether the odds have changed. First, what do we know? As the deck was chosen at random, we know that the probability of the card being drawn from the trick deck is the same as the probability of it being drawn from the normal one:

c02ue007

Here, T denotes the event that I was carrying a trick deck and Tc denotes the complementary event (not T) that I was carrying the normal deck.

We also know two conditional probabilities. The probability of drawing a black card from the trick deck is, of course, 1, while that of drawing a black card from a deck which has equal numbers of black and red cards is 1/2. In symbols, P(B|T) = 1 and P(B|Tc) is 1/2.

What we’d like to know is whether the two conditional probabilities P(T|B) and P(Tc|B) are equal. We begin by putting the two sets of facts we have together, using our conditional probability relation, P(B|T) = P(T and B)/P(T).

We know two of the values in the first relation, P(B|T) and P(T), and so we can solve for P(B and T) = P(B|T)P(T) = 1 × 1/2. Similarly,

c02ue008

Take another look at our Venn diagram in Figure 2.1. All the events in outcome B are either in A or in its complement Ac. Similarly,

c02ue009

We now know all we need to know to calculate the conditional probability P(T|B) for our conditional probability relation can be rewritten to interchange the roles of the two outcomes, giving

c02ue010

By definition P(Tc|B) = 1 − P(T|B) = 1/3 < P(T|B).

The odds have changed. Before I showed you the card, the probability of my showing you a black card was

c02ue011

When I showed you a black card, the probability it came from a black deck was 1/2 divided by 3/4 or 2/3!

Exercise 2.13: If R denotes a red card, what would be P(T|R) and P(Tc|R)?


A Too Real Example
Think the previous example was artificial? That it would never happen in real life? My wife and I just came back from a car trip. On our way up the coast, I discovered that my commuter cup leaked, but, desperate for coffee, I wrapped a towel around the cup and persevered. Not in time, my wife noted, pointing to the stains on my jacket.
On our way back down, I lucked out and drew the cup that didn’t leak. My wife congratulated me on my good fortune and then, ignoring all she might have learned had she read this text, proceeded to drink from the remaining cup! So much for her new Monterey Bay Aquarium sweatshirt.

2.3.1 Market Basket Analysis

Many supermarkets collect data on purchases using barcode scanners located at the check-out counter. Each transaction record lists all items bought by a customer on a single purchase transaction. Executives want to know whether certain groups of items are consistently purchased together. They use these data for adjusting store layouts (placing items optimally with respect to each other), for cross-selling, for promotions, for catalog design and to identify customer segments based on buying patterns.

If a supermarket database has 100,000 point-of-sale transactions, out of which 2000 include both items A and B and 800 of these include item C, the association rule “If A and B are purchased then C is purchased on the same trip” has a support of 800 transactions (alternatively 0.8% = 800/100,000) and a confidence of 40% (= 800/2000).

Exercise 2.14: Suppose you have the results of a market-basket analysis in hand. (1) If you wanted an estimate of the probability that a customer will purchase anchovies, would you use the support or the confidence? (2) If you wanted an estimate of the probability that a customer with anchovies in her basket will also purchase hot dogs, would you use the support or the confidence?

2.3.2 Negative Results

Suppose you were to bet on a six-horse race in which the horses carried varying weights on their saddles. As a result of these handicaps, the probability that a specific horse will win is exactly the same as that of any other horse in the race. What is the probability that your horse will come in first?

Now suppose, to your horror, a horse other than the one you bet on finishes first. No problem; you say, “I bet on my horse to place,” that is, you bet it would come in first or second. What is the probability you still can collect on your ticket? That is, what is the conditional probability of your horse coming in second when it did not come in first?

One of the other horses did finish first, which leaves five horses still in the running for second place. Each horse, including the one you bet on, has the same probability to finish second, so the probability you can still collect is one out of five. Agreed?

Just then, you hear the announcer call out that the horses are about to line up for the second race. Again there are six horses and each is equally likely to finish first. What is the probability that if you bet on a horse to place in the second race that you will collect on your bet? Is it 1/6 + 1/5?

There are three ways we can arrive at the correct answer when all horses are equal in all respects:

1. We could notice that the probability that our horse will finish second is exactly the same as the probability that it will finish first (or the probability that it will finish dead last, for that matter). As these are mutually exclusive outcomes, their probabilities may be added. The probability is 2/6th that your horse finishes first or second.
2. We could list all 6! mutually exclusive outcomes of the race and see how many would lead to our collecting on our bet—but this would be a lot of work.
3. Or, we could trace the paths that lead to the desired result. For example, either our horse comes in first with probability 1/6 or it does not, with probability 5/6. If it doesn’t come in first, it might still come in second with probability 1/5. The overall probability of your collecting on your bet is Pr{your horse wins} + Pr{your horse doesn’t win}*Pr{your horse is first among the nonwinning horses} = 1/6 + 5/6*1/5 = 2/6.

Exercise 2.15: Suppose 10 people are in a class. What is the probability that no two of them were born on the same day of the week? What is the probability that all of them were born in nonoverlapping four-week periods?

Exercise 2.16: A spacecraft depends on five different mission-critical systems. If any of these systems fail, the flight will end in catastrophe. Taken on an individual basis, the probability that a mission-critical system will fail during the flight is 1/10. (1) What is the probability that the flight will be successful?

NASA decides to build in redundancies. Every mission-critical system has exactly one back-up system that will take over in the event that the primary system fails. The back-up systems have the same probability of failure as the primaries. (2) What is the probability that the flight will be successful?

Exercise 2.17: A woman sued a Vegas casino alleging the following: she asked a security guard to hold her slot machine while she hit the buffet; he let somebody else use “her” machine while she was gone; that “somebody else” hit the jackpot; that jackpot was rightfully hers. The casino countered that jackpots were triggered by a random clock keyed to the 1/1000th of a second, thus, even had the woman been playing the machine continuously, she might not have hit the jackpot. How would you rule if you were a judge?

Exercise 2.18: In the United States in 1985, there were 2.1 million deaths from all causes, compared with 1.7 million in 1960. Does this mean it was safer to live in the United States in the 1960s than in the 1980s?

Exercise 2.19: You are a contestant on “Let’s make a deal.” Monty offers you a choice of three different curtains and tells you there is a brand new automobile behind one of them plus enough money to pay the taxes in case you win the car. You tell Monty you want to look behind curtain number 1. Instead, he throws back curtain number 2 to reveal … a child’s toy. “Would you like to choose curtain number 3 instead of number 1?” Monty asks. Well, would you?

2.4 INDEPENDENCE

A key element in virtually all the statistical procedures we will consider in this text is that the selection of one member of a sample takes place independently of the selection of another. In discussing the game of Craps, we assumed that the spots displayed on the first die were independent of the spots displayed on the second. When statistics are used, either we:

1. Assume observations are independent.
2. Test for independence
3. Try to characterize the nature of the dependence (Chapter 7).

Two events or observations are said to be independent of one another, providing knowledge of the outcome or value of the one gives you no information regarding the outcome or value of the other.

In terms of conditional probabilities, two events A and B are independent of one another providing that P(A|B) = P(A), that is, our knowledge that B occurred does not alter the likelihood of A. (This relation should be read as “the probability that A occurs given that B occurs is equal to the probability that A occurs.”)

We can use this relation to show that if A and B are independent, then the probability they will both occur is the product of their separate probabilities, P(A and B) = P(A)*P(B). For from the definition of conditional probability, P(A and B) = P(A)*P(A and B|A) = P(A)*P(B|A) = P(A)*P(B).

Warning: Whether events are independent of one another will depend upon the context. Imagine that three psychiatrists interview the same individual who we shall suppose is a paranoid schizophrenic. The interviews take place at different times and the psychiatrists are not given the opportunity to confer with each other either before or after the interviews take place.

Suppose now that these psychiatrists are asked for their opinions on (1) the individual’s sanity, and, having been informed of the patient’s true condition, (2) their views on paranoid schizophrenia. In the first case, their opinions will be independent of one another; in the second case, they will not.

Exercise 2.20: Can two independent events be mutually exclusive?

Exercise 2.21: Draw a Venn diagram depicting two independent events one of which is twice as likely to occur as the other.

Exercise 2.22: Do the following constitute independent observations?

a. Several students sitting together at a table are asked who their favorite movie actress is.
b. The number of abnormalities in each of several tissue sections taken from the same individual.
c. Opinions of several individuals whose names you obtained by sticking a pin through a phone book, and calling the “pinned” name on each page.
d. Opinions of an ardent Democrat and an ardent Republican.
e. Today’s price in Australian dollars of the German mark and the Japanese yen.

Exercise 2.23: Based on the results in the following contingency tables, would you say that sex and survival are independent of one another in Table A? In Table B?

Table A
Alive Dead
Men 15 5
Women 15 10
Table B
Alive Dead
Men 15 10
Women 12 8

Exercise 2.24: Provide an example in which an observation X is independent of the value taken by an observation Y, X is independent of a third observation Z, and Y is independent of Z, but X, Y, and Z are dependent.

2.5 APPLICATIONS TO GENETICS

All the information needed to construct an organism, whether a pea plant, a jelly fish, or a person, is encoded in its genes. Each gene contains the information needed to construct a single protein. Each of our cells has two copies of each gene, one obtained from our mother and one from our father. We will contribute just one of these copies to each of our offspring. Whether it is the copy we got from our father or the one from our mother is determined entirely by chance.

You could think of this as flipping a coin, one side says “mother’s gene,” the other side says “father’s gene.” Each time a sperm is created in our testis or an ovum in our ovary, the coin is flipped.

There may be many forms of a single gene; each such form is called an allele. Some alleles are defective, incapable of constructing the necessary protein. For example, my mother was rh–, meaning that both her copies of the rh gene were incapable of manufacturing the rh protein that is found in red blood cells. This also means that the copy of the rh gene I obtained from my mother was rh–. But my blood tests positive for the rh protein, which means that the rh gene I got from my father was rh+.

Exercise 2.25: The mother of my children was also rh–. What proportion of our children would you expect to be rh-?

Exercise 2.26: Sixteen percent of the population of the United States is rh–. What percentage do you expect to have at least one rh– gene? (Remember, as long as a person has even one rh+ gene, they can manufacture the rh protein.)

The gene responsible for making the A and B blood proteins has three alleles, A, B, and O. A person with two type 0 alleles will have blood type O. A person with one A allele and one B allele will have blood type AB. Only 4% of the population of the United States has this latter blood type.

Our genes are located on chromosomes. The chromosomes come in pairs, one member of each pair being inherited from the father and one from the mother. Your chromosomes are passed onto the offspring independently of one another.

Exercise 2. 27: The ABO and rh genes are located on different chromosomes and thus are inherited independently of one another. What percentage of the population of the United States would you expect to have the AB rh+ blood type?

2.6 SUMMARY AND REVIEW

In this chapter, we introduced the basics of probability theory and independence and applied the elements of both to genetics. We learned additional R commands for controlling program flow (if, else) and for displaying properties of binomial distributions. We also learned how to create our own special purpose R functions.

Exercise 2.28: Make a list of all the italicized terms in this chapter. Provide a definition for each one along with an example.

Exercise 2.29: (Read and reread carefully before even attempting an answer) A magician has three decks of cards, one with only red cards, one that is a normal deck, and one with only black cards. He walks into an adjoining room and returns with only a single deck. He removes the top card from the deck and shows it to you. The card is black. What is the probability that the deck from which the card came consists only of black cards?

Exercise 2.30: An integer number is chosen at random. What is the probability that it is divisible by 2? What is the probability that it is divisible by 17? What is the probability that it is divisible by 2 and 17? What is the probability that it is divisible by 2 or 17? (Hint: A Venn diagram would be a big aid in solving this last part.)

Exercise 2.31: Pete, Phil, and Myron are locked in a squash court after hours with only a Twinkie and a coin between them. The only thing all three can agree on is that they want a whole Twinkie or nothing. Myron suggests that Pete and Phil flip the coin, and that the winner flips a coin with him to see who gets the Twinkie. Phil who is a graduate student in statistics says this is unfair. Is it unfair and why? How would you decide who gets the Twinkie?

Exercise 2.32: People v. Collins,* concerned a prosecution of an African-American and his Caucasian wife for robbery. The victim testified her purse was snatched by a girl with a blond ponytail; a second witness testified he saw a blond girl, ponytail flying, enter a yellow convertible being driven by an African-American with a beard and mustache. Neither witness could identify the suspects directly. In an attempt to prove the defendants were in fact the persons who had committed the crime, the prosecutor called a college instructor of mathematics to establish that, assuming the robbery were committed by a Caucasian woman with a blond ponytail who left the scene in a yellow Lincoln accompanied by a African-American with a beard and mustache, there was an overwhelming probability the crime was committed by any couple answering to such distinctive characteristics.

Assume you were on the jury. What would you decide and why?

Notes

* I had a professor at Berkeley who wrote a great many scholarly articles on the subject of “coins that stand on edge,” but then that is what professors at Berkeley do.

* If A and B are mutually exclusive, P(A and B) = P(A) × P(B). See Section 2.2 for the proof of this statement.

* Curiously, not a single Venn diagram is to be found in John Venn’s text, The Logic of Chance, published by Macmillan and Co, London, 1866, with a 3rd edition in 1888.

* If you want to be precise, the probability of throwing a head is probably only 0.49999, and the probability of a tail is also only 0.49999. The leftover probability of 0.00002 is the probability of all the other outcomes—the coin stands on edge, a sea gull drops down out of the sky and takes off with it, and so forth.

* If, that is, the probability of survival is the same for every patient. When there are obvious differences from trial to trial—for example, one subject is an otherwise healthy 35-year old male, the other an elderly 89-year old who has just recovered from pneumonia—this simple binomial model would not apply.

 435 U.S. 223, 236–237 (1978).

 Strictly speaking, it is not the litigant but the potential juror whose rights might have been interfered with. For more on this issue, see Chapter 2 of Phillip Good, Applications of Statistics in the Courtroom (Chapman and Hall, 2001).

* 68 Cal. 2d 319, 66 Cal. Rptr. 497 (1968).