CHAPTER 30
Basic statistics: Assessing the impact of therapeutic interventions with odds-ratios, relative risk, and hazard ratios

Jesse J. Brennan and Edward M. Castillo

Department Emergency Medicine, University of California San Diego, San Diego, CA, USA

So you just had a brilliant idea for an intervention trial and you are feeling quite happy with yourself (as you should), but how will you determine if the intervention had any impact on the main outcome? Here we describe some simple but informative approaches to assess the impact of your intervention. By the end of this chapter, you will be able to answer the following questions:

What is relative risk?
What is an odds ratio?
Which one should I use?
Time as a factor: What is a hazard ratio?
How do you interpret statistical significance?

Before we dive in, let us first address your most pressing question: “Why in the non-statistical world must I be subjected to math”? Given the availability of popular statistical software programs with point and click options, as well as free on-line statistical calculators, that is a fair question. The answer is not so you will literally be able to do it by hand (or by calculator), because quite frankly you will likely never need to. However, we highlight the different methods by which each is calculated so that you might better understand the subtle differences between relative risks and odds ratios. To this end, we must both “do the math”.

Hypothetical study example

To illustrate how a relative risk and odds ratio is calculated, we will be referring to a hypothetical example of a randomized clinical trial targeting patients at risk for returning to the emergency department (ED) for repeated care (Table 30.1). Patients in the intervention group are treated with what we hope is the proverbial magic pill (or in this case a heavy dose of case management) while patients in the control group receive the standard of care. The outcome of interest is whether the patient returns to the ED for repeated care within 30 days of the initial visit (Yes or No).

Table 30.1 Number of patients with a 30-day ED visit by study group.

Study Group	Outcome (30-day ED Visit)
	Yes (not desired)	No (desired)	Total Patients
Intervention (enhanced case management)	15 (a)	85 (c)	100 (a + c)
Control (standard of care)	20 (b)	80 (d)	100 (b + d)

What is relative risk?

Simply put, relative risk is a ratio of two event probabilities. In the context of an intervention trial, we might say that it is the risk of an event (or outcome) occurring in the intervention group compared to the risk of that event occurring in the control group. However, determining the risk (or probability) of the event occurring in each group is not as complicated as you might imagine. For this purpose, probability is equivalent to a proportion. Thus, calculating the relative risk is “relatively” simple. We simply divide the proportion of patients in which the event occurred in the intervention group by the proportion of patients in which the event occurred in the control group. The resulting relative risk value can then be used to interpret both the directionality and magnitude of the relationship; that is, whether the risk of the event occurring in the intervention group is more or less likely to occur than in the control group, and to what extent. A value of one implies identical risk for both groups; whereas, a value smaller than one indicates less risk of an event occurring in the intervention group compared to the control, and a value larger than one indicates more risk. For example, a value of 0.5 would indicate half the risk; while a value of 2.0 would indicate twice the risk.

Referring to the example in Table 30.1, the relative risk can be calculated as the proportion of patients in the intervention group with a 30-day visit (15 ÷ 100 = 0.15) divided by the proportion of patients in the control group with a 30-day ED visit (20 ÷ 100 = 0.20) (Box 30.1). So we find that the risk of coming back to the ED within 30 days for patients receiving case management is lower than the risk (or 0.75 times the risk) for patients receiving the standard of care (0.15 ÷ 0.20 = 0.75) (Box 30.1). Simple as π.

What is an odds ratio?

In its simplest form, an odds ratio is a ratio of the odds of an event occurring in two different groups. In the context of an intervention trial, we might say that it is the odds of an event (or outcome) occurring in the intervention group compared to the odds of that event occurring in the control group. This may sound similar to the definition of relative risk, but in this context the odds of an event occurring and the probability of an event occurring are not derived in the same manner.

To calculate the odds ratio, we first determine the odds of a 30-day visit separately for each group. For example, referring to the example in Table 30.1, the odds of a 30-day ED visit in the intervention group is equivalent to the probability of the event occurring (15 ÷ 100 = 0.15) divided by the probability of the event not occurring (85 ÷ 100 = 0.85), or 0.18. Similarly, the odds of a 30-day ED visit in the control group is equivalent to the probability of the event occurring (20 ÷ 100 = 0.20) divided by the probability of the event not occurring (80 ÷ 100 = 0.80), or 0.25 (Box 30.2). Again, it is important to note here that the probability of the event occurring or not occurring is simply the proportion of patients in which the event does or does not occur. After we have determined the odds separately for each group, we can calculate the odds ratio by simply dividing the odds of a 30-day visit in the intervention group (0.18) by the odds of a 30-day visit in the control group (0.25). So we find that the odds of a patient returning to the ED within 30 days after receiving case management is less than (or 0.71 times) the odds of a patient returning to the ED within 30 days without case management (Box 30.2).

Similar to relative risk, interpretation of directionality and magnitude of the relationship also centers on the value of one. A value of one implies that the odds of an event occurring are identical for both groups; whereas, a value smaller than one indicates lower odds of an event occurring in the intervention group compared to the control, and a value larger than one indicates higher odds.

Which one should I use?

Generally, an odds ratio can be used in any type of study design whereas relative risk is reserved for either retrospective or prospective cohort studies. For instance, it is often preferable to use relative risk in a cohort study in which we might compare similar groups of patients who have either been exposed or not exposed (to treatment) in relation to the occurrence of an event or outcome. For example, if the goal is to simply describe which group is more or less likely to have a 30-day ED visit in relation to patient exposure to the intervention, the relative risk is straight forward and often easier to interpret than the odds ratio. Intuitively, it is just easier to speak to the likelihood of a 30-day visit than it is to the odds.

However, there is a specific type of study design, commonly referred to as retrospective case–control design, in which the use of relative risk is generally ill-advised. Imagine, for example, we wish to study potential risk factors for obesity in adulthood, such as being overweight as a child. Using a case–control design to address this question, we might first select an adult group of clinically obese patients and an adult group of non-obese patients, then assess the number of patients in each group who were overweight as early as five years of age. In such a design, in which group membership has been assigned based on the outcome rather than exposure, the probability of obesity (or non-obesity) in adulthood is more of a reflection of the number of patients assigned to each group than anything else, which simply makes the relative risk uninformative.

Finally, regardless of which type of study design is used, it is preferable (easier) to use an odds ratio if there is a need to account for (control for) additional factors (confounders) that might be related to the outcome. In such a case, more complex statistical analyses are employed, such as multiple logistic regression analyses. The resulting odds ratios are referred to as adjusted odds ratios, as they describe the independent association with the outcome with consideration to other covariates in the model.

So at this point you might be thinking: “The relative risk and odds ratio in this example are almost identical. Does it really matter which one I use?” Keep in mind that an odds ratio is often used to estimate true risk when true risk cannot be directly estimated; it is an approximation of relative risk. It just so happens that in most cases involving trials in a clinical setting, the outcomes under study are relatively infrequent (death, adverse event, etc.), in which case there is little difference, either literally or theoretically, between an odds ratio and a relative risk. However, the larger the initial probability of the outcome (i.e., the larger the proportion of patients with 30-day ED visits), the more dissimilar the odds ratio will be to the relative risk. For example, using the example in Table 30.1, consider that the proportion of patients with a 30-day ED visit was 0.60 in the intervention group and 0.80 in the control group (instead of 0.15 and 0.20, respectively). The relative risk would still be equal to 0.75 (0.60 divided by 0.80); however, the corresponding odds ratio would be approximately 0.38 (the odds of 30-day visit in intervention [0.60 divided by 0.40 = 1.50] divided by the odds of 30-day visit in control group [0.80 divided by 0.20 = 4.00]). In such a case, the odds ratio becomes overinflated in relation to the relative risk.

Regardless of which statistic you decide to use, this last point is critical to understanding how assuming that the relative risk and odds ratio are equivalent to one another can lead to confusion and misinterpretation. From a clinical perspective, it would be appropriate to interpret the relative risk estimate of 0.75 to mean that the intervention reduced the risk of a 30-day visit by 25% (1 – relative risk). However, based on the corresponding odds ratio of 0.38, it is not appropriate to say that the intervention reduced the risk of a 30-day visit by 62% (we just determined that it is 25%). The confusion arises in the literature, for example, when an odds ratio is described as an outcome being twice as likely to occur in group A as group B, rather than twice the odds. Based on this language, we are likely to infer that researchers are actually describing relative risk, which we know is not the same. Although, even in such a case, if the relative risk and odds ratio were similar, are we really at terrible risk of reaching a different conclusion in our study (i.e., the intervention did or did not have an impact)? No, not really. We are likely to reach the same conclusion even if they are not similar. However, the more dissimilar these estimates are the more caution we need to take in interpreting an odds ratio in relation to effect size; that is to say, the impact of the intervention.

Time as a factor: What is a hazard ratio?

Now that we have described the difference between the relative risk and odds ratio, it is worth mentioning an additional approach that can be used when studying the timing of the event in addition to whether the event simply occurred or did not occur. For example, say we are not only interested in the risk of repeat ED visits within 30 days, but we are also interested in this risk over time; such as time to a repeat ED visit as measured by 3, 7, 14, 21, and 30 days from the initial visit. In this instance, we can examine a cumulative risk associated with each time point as well as relative risk for the entire study period taking time to outcome into consideration. This is what is referred to as the hazard ratio. While calculation of the hazard ratio is beyond the scope of this chapter, it can be generated using Cox proportional hazard regression analyses provided by most statistical software programs. In addition, you can visually inspect the relative proportion of patients who do and do not have a repeat ED visit over time (survival if you will) using a Kaplan-Meier plot. If, for example, the time to a 30-day ED visit for patients who received case management was longer than patients who received standard of care, it will be apparent on the plot and the hazards ratio will likely reflect a value greater in magnitude than our original relative risk obtained at 30-days, taking time to 30-day visit into consideration (Figure 30.1).

c30-fig-0001 — **Figure 30.1** Hypothetical survival curves of time to repeat ED visit.

How do you interpret statistical significance?

The final piece to this puzzle is to determine whether the relative risk (or odds ratio) we find can be considered meaningful in the context of our intervention. To interpret statistical significance, we rely on the 95% confidence interval (CI). Essentially, if the 95% CI includes the value of one, then the estimate is not statistically significant, meaning that the probability of finding a relative risk (or odds ratio) of this magnitude by chance alone is greater than 5% (see also Chapter 29). For instance, in our study example shown in Table 30.1, the 95% CI for a relative risk of 0.75 includes the value of 1.0 (0.41–1.38). In this case, we would conclude that our intervention did not significantly impact 30-day ED visits, or that the risk of a 30-day ED visit for patients receiving case management was equivalent to patients who received standard of care. However, holding constant all other factors that contribute to a methodologically sound study, keep in mind that statistical significance is related to the power of a study to find a treatment effect if it truly exists. For example, if sample size is very large, a relative risk (or odds ratio) close to 1.0 may be considered statistically significant, but it does not necessarily mean it is clinically relevant. Similarly, if sample size is very small a relative risk much higher (or lower) than 1.0 may not be considered statistically significant, but it does not necessarily mean that the treatment had no effect (see also Chapter 28).

Unless you are a contestant on a television game show, “p value <0.05” does not have to be your final answer. The p value simply informs us of the likelihood that the difference (or effect size) we are seeing between two groups may or may not be due to chance alone. However, if you have planned ahead of time you should already know whether your sample size is sufficient to detect a meaningful difference, and how large of an effect size you would need to reach statistical significance. If you do not have a lot of experience with statistics, do not be shy about consulting with someone who does before you begin collecting data. It will not matter how much data you collect or how well you collect it if in the end you cannot properly evaluate your study.

Final thoughts

Hopefully you will leave this chapter with a better understanding as to why the odds ratio and relative risk are not interchangeable and should not be interpreted as such, as well as how and when to use these tools to describe the results of your study. If not, you can always ask a statistician to do it!