A provocative question, a clever operationalization, and a powerful research design do not amount to much without people who are willing to give researchers access to their personal lives. In research on human beings, the people who provide data are called the sample because they are a subset of a broader population that, theoretically, could have provided very similar data. Identifying exactly who provided data is another important part of evaluating whether the results of a study generalize—that is, whether a study has external validity. Unlike research in chemistry or physics (where an atom is an atom wherever you go), research on people must accept the fact that those who provide data may differ from people who do not provide data in ways that may skew the results.
Suppose a researcher wants to understand the different reasons people remain in their intimate relationships. Some people might stay because they value their partner; others might stay because they share children or resources (like a house or a car) with their partner, and it would be too costly to leave. Which kind of reason is more important? The answer might depend largely on who is being asked. College students do not generally sign mortgage contracts or have children with their dating partners. If the sample consists entirely of college students, the researcher might conclude that satisfaction with the relationship is the only thing that keeps couples together. Among married couples, on the other hand, spouses often do share resources they would lose by ending the relationship. If the sample includes married couples, the researcher might reach very different conclusions. In both cases, the samples must be analyzed carefully to determine whether the results are likely to apply to other groups.
Because researchers want their studies to have high external validity, it’s ideal to collect representative samples—samples consisting of people who are similar to the population to which the researchers would like to generalize. A sample consisting entirely of college students in dating couples at a particular university, for example, might be representative of other college student dating couples at that university. Most of the time, however, researchers want their results to apply more broadly so they can draw conclusions that apply to a wider range of intimate relationships. How can researchers decide if a given sample is representative or not?
To address this issue, David Sears (1986) argued that the external validity of a study is threatened only by differences between the sample and the relevant population on dimensions that could conceivably affect the study results. In other words, if a researcher studies only newlywed marriages, the results may not apply to well-established marriages or same-sex couples, because it is easy to imagine potentially important ways the relationships of newlyweds may differ from those of the other two groups (e.g., shared resources, level of commitment, relationship duration, etc.). On the other hand, if a researcher studies only people with freckles, the results may well apply to other people as well, because no one has yet argued that having freckles affects relationship processes. (Box 3.3 presents more about the complexities of studying couples.)
Box 3.3
Spotlight on . . .
The Challenges of Studying Couples
An irony of research on intimate relationships is that most of the data are collected solely from individuals (Furman,1984; Karney & Bradbury, 1995). Studying individuals is no easy task, but studying couples raises unique issues that more than double the complexity. Consider the challenges that researchers interested in couples must face.
How do you know that two people are a couple?
Married couples usually know whether or not they are married—the license, the rings, and the wedding are a dead giveaway. But expand the focus to unmarried couples and the boundaries that define “couplehood” get a lot fuzzier. For example, the National Longitudinal Survey of Adolescent Health had a large sample of high school students list all the people with whom they had been in a romantic relationship. Because the survey included all the students in several schools, the researchers also had data from most of the people that their respondents listed, allowing them to determine how often two people agreed they’d been in a romantic relationship with each other (Kennedy, 2006). How often did they agree? Less than half the time! Not surprisingly, better data on relationships come from pairs that agreed on the status of their relationship.
Whose information do you trust?
You might expect that two people in a couple would agree about the concrete details of their relationship, such as when they met, how long they’ve been together, and whether they have kids. They usually do, but not always. Sometimes partners differ in how they understand their relationship, in how they understand a particular question, or in how honest they want to be. All these possibilities lead to the same result: The researcher has two different answers to the same question about the same relationship. Deciding which one to trust can be a problem. If the question is about a quantity (such as relationship duration), one option is to take the average of the two answers. Other times the researcher may decide that one partner is inherently more trustworthy (e.g., women are quicker to report a pregnancy than men are, for obvious reasons).
Which effects do you care about?
Suppose you want to know about how the quality of a relationship is associated with personality. If you are studying individuals, you might ask people to report on their personality and their relationship satisfaction, and then estimate the correlation between their answers. Add another partner to your sample, however, and the possible associations multiply. The Actor-Partner Interdependence Model (Figure 3.8) points out all of the different ways two variables may be associated with each other within a couple (Cook & Kenny, 2005). To extend the personality example, each person’s personality may be associated with his or her own relationship satisfaction (the Actor effect). Each person’s personality may also be associated with his or her partner’s relationship satisfaction (the Partner effect). Finally, analyses of couples have to account for the fact that partners within a relationship tend to have similar personalities. Statistical techniques can tease all of these effects apart.
FIGURE 3.8The Actor-Partner Interdependence Model: possible associations of two variables in a couple. (Source: Adapted from Cook & Kenny, 2005.)
By recognizing and facing the unique challenges of studying couples, researchers have developed methods customized for this field. These strategies help characterize relationship science as an established discipline.
Paying careful attention to study participants is critical, because a lot of research on intimate relationships has been conducted on a pretty narrow range of people. One review of 280 studies published in the Journal of Social and Personal Relationships found that over half of all the research sampled exclusively from college students (de Jong Gierveld, 1995). Why? Not because researchers have an abiding interest in college students, but because they are easy to find on the campuses where much of this research takes place. They are referred to as convenience samples. When composed of college students, convenience samples are more likely to be middle class and less likely to be married than other possible samples. Data from these samples may help researchers understand dating relationships among educated young people, but they may reveal little about relationships in later life or relationships among people who do not go to college. The same goes for studies of married couples. In fact, 75 percent of longitudinal research on marriage has involved convenience samples that were primarily white, Protestant, and middle class (Karney & Bradbury, 1995). Again, it’s not clear to what extent these results generalize to marriages in other cultures, religions, or socioeconomic groups.
Throughout this book, we make every effort to identify and describe research that explores the full diversity of intimate relationships across cultures and around the world. However, as just noted, this kind of truly global research is still rare. For most research, obtaining a representative sample remains an elusive goal. Until information on a broader range of couples has accumulated, careful researchers must take pains to draw conclusions appropriate only to the samples they have studied.
MAIN POINTS
Even when the study design is perfectly appropriate to the questions being asked, conclusions can be limited by the nature of the sample, the portion of the population that actually provides the data.
Researchers try to collect data from people who represent the populations to which they would like their results to generalize, but this is hard to accomplish.
Most relationship science research relies on convenience samples. Even if the sample is not representative of the population, it can at least be shown to be similar to the population on dimensions that might affect the study results.
To the extent that a sample providing data is different from a population on those dimensions, any conclusions that apply to the sample may not apply to other groups.
