After Chapter 11.5, you will be able to:
Until this point, we’ve been discussing research in a vacuum but our goals with any research are application-based. In order to apply the data generated, there are practical concerns that we must consider. For example, we must take into account the statistical strengths and weaknesses of a study, especially those that are related to the differences between the target population and the study sample. We also consider ways in which bias impacts the ability to use study conclusions in the real world, and whether there is any true justification for an intervention.
In statistics and research, we generally work with a sample rather than an entire population. A population is the complete group of every individual that satisfies the attributes of interest. Populations may be very large; for example, the population of humans is over seven billion people. In contrast, a population with a large number of qualifiers—for example, the population of American females between 18 and 30 years old who have Darier’s disease, a rare skin condition—will be much smaller (in this case, about 100 people). Information that is calculated using every person in a population is called a parameter.
Working with a population is generally not feasible, even for smaller groups. Therefore, we make generalizations about populations based on sample data. A sample is any group taken from a population that does not include all individuals from the population. Ideally, samples will be representative of the population, and there are several methods of ensuring this. Random samples are generally considered the gold standard, although selecting for certain small subgroups may also be used. Information about a sample is called a statistic. With comparatively large or repeated samples, statistics can be used to estimate population parameters. If only a single small sample is taken, then very little information can be gleaned about the population.
When analyzing a study, we also look for markers of internal validity (or support for causality as discussed earlier) and external validity, or generalizability. Studies with low generalizability have very narrow conditions for sample selection that do not reflect the target population, whereas studies with high generalizability have samples that are representative of the target population. For example, a psoriasis study with low generalizability might have only participants who were diagnosed within the last year, while a study with high generalizability would have participants with a distribution of time since diagnosis that is similar to the population of all psoriatic patients.
Drugs undergo continuous evaluation in part because of poor preclinical generalizability. Some marketing changes or additional warnings may become necessary, or a drug may even be taken off the market. These are unforeseen risks or outcomes that only become apparent when the drug becomes available to the entire population.
As future doctors, we are interested in applying research to our patients. To do so, we’ll need to consider whether the data is sufficient for the recommendation or exclusion of any therapy or treatment plan.
In research, the primary marker of success is being able to generate results that are statistically significant—that is, not the result of random chance. However, even the smallest difference between two treatments may be significant mathematically. For example, a decrease in systolic blood pressure of one millimeter of mercury could be statistically significant; however, it is not likely to change patient outcomes. In this way, we must assess whether there is clinical significance—a notable or worthwhile change in health status as a result of our intervention.