<!DOCTYPE html
  PUBLIC "-//W3C//DTD XHTML 1.1//EN" "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml"><head><title>Mixed Methods Research:</title><link rel="stylesheet" type="text/css" href="styles/Chapter.css" /><link href="http://purl.org/dc/elements/1.1/" rel="schema.DC" /><meta http-equiv="Content-Type" content="text/html; charset-utf-8" /><meta name="DC.creator" content="Ghosh, Rajashi" /><meta name="DC.identifier" content="978-1-7998-2460-2 DOI: 10.4018/978-1-7998-2460-2.ch079" /><meta name="DC.title" content="Mixed Methods Research: Mixed Methods Research" /><meta name="DC.description" content="Mixed methods research (MMR) is increasingly becoming a popular methodological approach in several fields due to the promise it holds for comprehensive understanding of complex problems being researched. However, researchers interested in MMR often lack reference to a guide that can explain the key issues pertaining to the paradigm wars influencing MMR, different objectives of MMR, choice of MMR designs, and articulation of research questions in MMR. This paper addresses that gap through providing a peek into these issues through illustrative examples. This brief introduction to MMR is meant to encourage readers to delve deeper into the MMR literature and make informed decisions in designing and implementing MMR studies." /><meta name="DC.publisher" content="IGI Global" /><meta name="DC.type" content="chapter" /><meta name="DC.format" content="electronic" /><meta name="DC.source" content="http://services.igi-global.com/resolvedoi/resolve.aspx?doi=10.4018/978-1-7998-2460-2.ch079" /><meta name="DC.language" content="English" /><meta name="DC.coverage" content="" /><meta name="DC.rights" content="Access limited to members" /></head><body><h1 class="SectionChapterNumber">Chapter 79</h1><h1 class="title">Mixed Methods Research:</h1><p class="subtitle">What are the Key Issues to Consider?</p>

<ul class="affiliates"><li class="affiliates">Rajashi Ghosh<br />Drexel University, Philadelphia, USA</li></ul><div><p class="subhead_1">ABSTRACT<a class="ThirdLevelTocAnchor" id="ach079sub1"></a></p><p class="abstract">Mixed methods research (MMR) is increasingly becoming a popular methodological approach in several fields due to the promise it holds for comprehensive understanding of complex problems being researched. However, researchers interested in MMR often lack reference to a guide that can explain the key issues pertaining to the paradigm wars influencing MMR, different objectives of MMR, choice of MMR designs, and articulation of research questions in MMR. This paper addresses that gap through providing a peek into these issues through illustrative examples. This brief introduction to MMR is meant to encourage readers to delve deeper into the MMR literature and make informed decisions in designing and implementing MMR studies.</p></div>


<p class="subhead_1">INTRODUCTION<a class="ThirdLevelTocAnchor" id="ach079sub2"></a></p>
<p class="Para_No_Ind">Mixed Methods research is a type of research where a single researcher or a team of researchers mixes the different elements of quantitative and qualitative research methods for the purpose of achieving greater breadth and depth of understanding of the topic being researched (Johnson, Onwuegbuzie, &amp; Turner, 2007). It is very important to understand here that the word “method” implies more than just data collection methods. In addition to methods of data collection (e.g., surveys, interviews, observation etc.), the word “methods” in Mixed Methods includes “methods of research (e.g., experiments, ethnography), and related philosophical issues (e.g., ontology, epistemology, axiology)” (Johnson et al., 2007, p. 118). It is this realization that helps to distinguish multimethod research from mixed method research. The notion of multimethod research was first championed by Campbell and Fiske (1959) for the purpose of triangulation. They advocated using more than one data collection measure to ensure that the explained variance can be attributed to the phenomenon being studied and not to a particular measure being employed. This provides evidence for the validity of the results and helps to counteract the possibility that the results are only methodological artifacts.</p>
<p class="Para_Ind">While the practice of collecting more than one kind of data through multiple data collection measures provided the impetus for researchers to think about the merits of combining different data collection methods, the use of multiple measures of data collection alone is not sufficient to conduct Mixed Methods Research (MMR). To conduct MMR, researchers need to carefully consider where they are positioned in regards to the paradigm wars between positivism/post-positivism largely guiding quantitative research and interpretivism largely guiding qualitative research, philosophical affiliation to pragmatism as an alternative to positivism/post-positivism and interpretivism, and the multitude of options for mixed methods research designs.</p>
<p class="Para_Ind">This paper will provide a brief peek into those considerations. First, I will present an account of how the paradigm wars between the quantitative and qualitative research camps have shaped the field of MMR and led to the evolution of alternative frameworks including pragmatism as the most common guiding approach or philosophy of MMR. Second, I will provide a discussion of the different purposes and reasons of conducting MMR leading to different types of MMR designs. Finally, I will conclude with explaining how to frame research questions in MMR.</p>


<p class="subhead_1">PARADIGM WARS AND MMR<a class="ThirdLevelTocAnchor" id="ach079sub3"></a></p>
<p class="Para_No_Ind">Before I discuss the different paradigms guiding the quantitative and qualitative research streams and how they might or might not be in contention when it comes to MMR, it is important to first understand what we mean by the term “paradigm”. Broadly, paradigms can be defined as “shared belief systems that influence the kinds of knowledge researchers seek and how they interpret the evidence they collect” (Morgan, 2007, p. 50). A closer look though would reveal subtle differences in the ways in which the term “paradigm” has been understood and used to guide research. Morgan (2007) reviews four basic versions of the paradigm concept and explains how accepting one version over the other might persuade us to support the combination of paradigms and reject the assumption that paradigms guiding quantitative and qualitative research are fundamentally incompatible. Out of the four versions, the two that are most relevant to understanding the paradigm wars and its implications for MMR are: (1) paradigms as epistemological stances, and (2) paradigms as shared beliefs among members of a specialty area (Morgan, 2007).</p>
<p class="Para_Ind">The epistemological stance approach to paradigms has hugely influenced the debate about whether it is possible to merge quantitative and qualitative research methods (Morgan, 2007; Tashakkori &amp; Teddlie, 2003). This approach considers paradigm to be “a deeper philosophical position relating to the nature of social phenomena and social structures” (Feilzer, 2010, p. 7). It includes ontological assumptions about the nature of reality, epistemological assumptions about the relationship between the researcher and the reality to be known, and methodological assumptions about the methods of generating knowledge about reality. This notion of paradigm got most traction in the debate on combining paradigms because the familiar triology of the concepts of ontology, epistemology, and methodology created by Guba and Lincoln (1988) was central to comparing the different paradigms with the most dominant paradigm of the time, positivism. As noted by Morgan (2007), the major advantage of this triology was that “it reduced positivism to the status of just one among a series of competing “paradigms” in social science methodology” (p. 59). Comparing and contrasting the paradigms in the interpretivist camp (e.g., constructivism, critical theory) with the dominant paradigm of positivism allowed the qualitative researchers to argue for the legitimacy of qualitative research which was essentially guided by the interpretivist paradigms. Unlike positivism that holds the ontological position that there is only one objective reality, the epistemological notion that researcher and the topic being researched are independent entities, and the methodological aim of measuring causal relationships between variables within a value-free framework, interpretivism espouses that reality is socially constructed and hence, there are multiple realities (ontology), researcher and the object of research are assumed to be interactively linked so that the findings are value-mediated (epistemology), and that a dialectical exchange between the researcher and the subjects is the primary means of inquiry (methodology).</p>
<p class="Para_Ind">However, in differentiating qualitative research from quantitative research and arguing for its legitimacy on the basis of the distinctiveness of the ontological, epistemological, and methodological assumptions of interpretivist paradigms, supporters of qualitative research made it very difficult for the proponents of MMR to argue their case of combining quantitative and qualitative research. If researchers perceive qualitative research to be fundamentally different from quantitative research, then it is difficult to see how the paradigmatic assumptions could be reconciled to justify mixing the two approaches in a study. What further complicated this challenge was that Guba and Lincoln (1988) took a top-down approach to their triology of ontology, epistemology, and methodology. In other words, even though they supposedly gave equal weight to ontology, epistemology, and methodology, the top-down orientation of their framework emphasized that the ontological assumption of reality (objective single reality in positivism vs. multiple socially constructed realities in interpretivism) was meant to guide a researcher’s epistemological assumptions (the researcher and the reality being independent or interactively linked) which in turn guided one’s methodological assumptions (the methods of generating knowledge about reality being value free or value mediated) (Morgan, 2007). Given this superior status of ontology, incompatibilities at the ontological level between the assumptions about the objective and subjective versions of reality dictated the impossibility of combining quantitative and qualitative research approaches, especially at the paradigmatic level.</p>
<p class="Para_Ind">Proponents of MMR sought a way out of this conundrum in primarily two different ways. They asked two fundamental questions: “First, do paradigms or mental models or other forms of philosophical assumptions matter in making inquiry decisions? If so, then what about paradigms most importantly matters, and how does it matter? Second, if not, then what else matters in inquiry decisions?” (Greene &amp; Caracelli, 2003, p. 96). The different frameworks proposed for mixing qualitative and quantitative research fall in either of these two stances. Thus, if one thinks that paradigms matter, then they can adopt a dialectic framework or a new paradigm framework. Greene, Bejamin, &amp; Goodyear, 2001). A dialectic framework considers the difference between paradigms to be an opportunity instead of a constraint. Mixed methods researchers using this framework intentionally integrate paradigms because they believe that a conversation between different paradigms can lead to “more comprehensive, insightful and logical results than either paradigm [interpretivist or postpositivist] could obtain alone” (Greene &amp; Caracelli, 1997, p. 10). A new paradigm framework incorporates a broader set of beliefs and assumptions about the nature of social reality and the knowledge about that reality. For example, mixed methods researchers adopting this framework may advocate the new paradigm of scientific realism which advocates that social reality is both causal and contextual thereby justifying the use of quantitative research and qualitative research to grasp the full scope of that reality (Greene &amp; Caracelli, 2003; Putnam, 1990).</p>
<p class="Para_Ind">Now, if one thinks that paradigms do not matter critically in making research inquiry decisions, they can adopt a substantive theory driven approach or a pragmatic approach (Greene et al., 2001). In a substantive theory driven approach, mixed methods researchers let the nature of concepts being studied and the theory underlying that concept decide which methods they will use and why. Therefore, the epistemological and ontological beliefs of the theory informing the researcher’s understanding of the concept guides the choice of how the researcher wants to study the concept, not the broad philosophical assumptions of any paradigm that is not specific to the concept being studied. Finally, the pragmatic approach suggests that the limitations and opportunities of a research context and the research questions should decide which methods to use when studying a topic instead of abstract philosophical assumptions of any paradigm (Greene et al., 2001). Pragmatists believe in existential reality (Dewey, 1925), which is “a reference to an experiential world with different elements or layers, some objective, some subjective, and some a mixture of the two” (Feilzer, 2010, p. 8). Among the multiple frameworks advanced to justify mixing of qualitative and quantitative research (Onwuegbuzie, Johnson, &amp; Collins, 2009), the framework most commonly used in MMR is the pragmatic approach (Teddlie &amp; Tashakkori, 2009). Quintessential to the pragmatic approach is its focus on shared beliefs and actions among researchers with different frames of reference. This focus is congruent with Morgan’s (2007) recommendation about thinking of paradigms as shared beliefs among members of a specialty area instead of epistemological stances. Unlike the claims of incommensurability advanced by the epistemological stance on paradigms guiding quantitative and qualitative research approaches:</p>

<p class="Para_Quote">A pragmatic approach would deny that there is any a priori basis for determining the limits on meaningful communication between researchers who pursue different approaches to their field. Instead, a pragmatic approach would place its emphasis on shared meanings and joint action. In other words, to what extent are two people (or two research fields) satisfied that they understand each other, and to what extent can they demonstrate the success of the shared meaning by working together on common projects? (Morgan, 2007, p. 67)</p>

<p class="Para_Ind">Furthermore, Morgan clarified that the pragmatic approach does not negate the relevance of epistemology. Instead, it discards the top-down approach that privileged ontological assumptions of reality and epistemological beliefs about the relationship between the researcher and the reality over methodology. A pragmatic approach re-directs our focus to the research questions and positions methodology (assumptions about the methods of generating knowledge about reality to address the research questions) as “an area that connects issues at the abstract level of epistemology and mechanical level of actual methods” (Morgan, 2007, p. 68) of data collection. The bottom line in pragmatism is that “research approaches should be mixed in ways that offer the best opportunities for answering important research questions” (Johnson &amp; Onwuegbuzie, 2004, p. 15). As noted by Johnson et al., (2007), “pragmatism offers a logic (i.e., use the combination of methods and ideas that helps one best frame, address, and provide tentative answers to one’s research question[s])” (p. 125) for mixing quantitative and qualitative research.</p>
<p class="Para_Ind">In addition, pragmatism also provides a reasonable middle ground between the seemingly opposite views on deduction and induction, objectivity and subjectivity, and context specificity and generalizability in quantitative and qualitative research (Morgan, 2007). A pragmatic approach can enable mixed method researchers to move back and forth between inductively converting observations and data into theories and then deductively testing those theories through action, i.e. to rely on abductive reasoning. It can justify the need felt by the researchers to embrace both objective and subjective frames of reference depending on the research question and thus take on an intersubjective approach. And, it can help researchers to challenge the myth that their results can either be too unique to only fit the context being studied or too generalizable that it can apply to all contexts irrespective of culture or history by deferring to the concept of transferability that advocates a careful consideration of factors which determine the extent to which the results of a research are context bound or generalizable. The notions of abductive reasoning, intersubjectivity, and transferability are the hallmark characteristics of a pragmatic approach which further bridges the two worlds of quantitative and qualitative research. Especially, the concept of intersubjectivity is the most quintessential pragmatic response to the argument of incompatibility between paradigms because it acknowledges that one can have multiple frames of reference based on the research question being asked. In doing so, pragmatism suggests that “there is no problem with asserting both that there is a single real world and that all individuals have their own unique interpretations of the world” (Morgan, 2007, p. 72). Instead of considering incommensurability as an all-or-nothing barrier, the pragmatists emphasize on “creating knowledge through joint actions or projects that different groups of people can accomplish together” (p. 72) with a reflexive orientation towards the social processes that produce both consensus and conflict between multiple frames of references (Morgan, 2007).</p>


<p class="subhead_1">PURPOSES OF CONDUCTING MMR AND MMR DESIGNS<a class="ThirdLevelTocAnchor" id="ach079sub4"></a></p>
<p class="Para_No_Ind">Researchers may opt to conduct MMR to benefit from the complementary strengths of quantitative and qualitative research so that the product is superior to a mono-method study (Johnson &amp; Turner, 2003). Most importantly, MMR can contribute to theory building in ways that a mono-method study cannot accomplish (Eisenhardt, 1989). As noted by Mintzberg (1979):</p>

<p class="Para_Quote">For while systematic data create the foundation for our theories, it is the anecdotal data that enables us to do the building. Theory building seems to require rich description, the richness that comes from anecdote. We uncover all kinds of relationships in our hard data, but it is only through the use of this soft data that we are able to explain them. (p. 587)</p>

<p class="Para_Ind">As described in this quote, MMR enables researchers to use both hard quantitative data and soft qualitative data to substantiate theoretical constructs and bolster evidence indicating patterns and themes leading to robust theory building research. In fact, MMR’s emphasis on abductive reasoning (i.e., interplay between inductive and deductive) enables mixed method researchers to engage in iterative cycles of theory building and testing unlike researchers conducting mono-method studies. Thus, it is hard to challenge that MMR is beneficial for theory building and testing and hence, indispensable for any field that wants to re-invigorate through discovering new concepts and notions. But, is that sufficient to guide researchers to understand why MMR can be suitable to address their research objectives that may not explicitly claim either theory building or testing? A list of specific motives for conducting MMR might help in that regards.</p>
<p class="Para_Ind">Greene, Caracelli, and Graham (1989) found five major reasons for conducting MMR studies: (a) triangulation (i.e., seeking convergence and corroboration of results from quantitative and qualitative methods); (b) complementarity (i.e., quantitative and qualitative methods studying overlapping but different facets of the same phenomenon); (c) initiation (i.e., discovering paradoxes and contradictions in results from quantitative and qualitative methods); (d) development (i.e., using the findings from one method to help inform the other method); and (e) expansion (i.e., seeking to expand the breadth and range of research by using the quantitative and qualitative methods for different inquiry components). While these reasons are quite compelling, it would help to map them in some way to the principles anchoring the plethora of designs available in MMR. That way, researchers can consciously choose designs that are most suitable for addressing their research objectives of triangulation, complementarity, initiation, development, and expansion.</p>
<p class="Para_Ind">A brief peek into the MMR literature reveals that there are numerous designs to choose from (Creswell, Plano, Guttmann, &amp; Hanson, 2003; Greene &amp; Caracelli, 1997; McMillan &amp; Schumacher, 2001; Maxwell &amp; Loomis, 2003; Morse, 2003; Teddlie &amp; Tashakkori, 2006). Although reflective of the diversity and richness in mixed methods studies, such variety of research designs can be quite confusing to beginning researchers or even to experienced researchers who are new to MMR. Given this, Leech and Onwuegbuzie, (2009) conducted a content analysis of the available mixed methods designs and offered a three-dimensional typology to guide researchers conducting mixed methods studies. The three dimensions advanced were: (1) time orientation (i.e., the quantitative and the qualitative strands occurring concurrently vs. sequentially); (2) emphasis of approaches (i.e., the quantitative and the qualitative strands having equal emphasis vs. one of the strands having a dominant emphasis); and (3) level of mixing (i.e., fully mixed where the quantitative and the qualitative strands are mixed within and across multiple components of a research study such as research objective, data collection, data analysis, and data interpretation vs. partially mixed where the quantitative and the qualitative strands are completed in their entirety and the mixing happens only at the data interpretation stage). In the following sections I would like to highlight how the three dimensions of time, emphasis, and level of mixing informing those designs might associate with the five reasons recommended for conducting mixed methods studies. I will do so with the help of hypothetical examples of research studies on the topics that interest me as a researcher.</p>

<p class="subhead_2">Time Orientation and MMR Purpose</p>
<p class="Para_No_Ind">In regards to the dimension of “time orientation”, how will a researcher decide if they want to conduct the quantitative and the qualitative strands concurrently or sequentially? Referring back to the five reasons for conducting mixed methods studies, it would make sense to conduct the strands concurrently if the researcher is interested in seeking triangulation, initiation, complementarity, and/or expansion. For example, if I were interested to study employees’ learning in formal mentoring programs, I can choose to conduct a survey measuring employees’ perceptions of learning in the mentoring program and also simultaneously interview a representative sample of employees about their experiences of learning and their thoughts about the mentoring program. Doing so would enable me to examine if the quantitative results from the survey and the qualitative results from the interview converge or corroborate one another (triangulation), if there are discrepancies between the realities reflected by the two kinds of data that can help me to re-frame my research question (initiation), if the qualitative and the quantitative findings complement each other by identifying something unique (complementarity), and if the quantitative data and the qualitative data expand my understanding of the mentoring program and its potential to impact employees’ learning by shedding light on different components of the program, i.e., program features and outcome (expansion). For examples of concurrent mixed method studies, see Hoffman and Nadelson (2010) and Feldon, Maher, Hurst, and Timmerman (2015).</p>
<p class="Para_Ind">Similarly, it would make sense to conduct the strands sequentially if the researcher is interested in the purpose of development. For instance, referring to the same example of a study exploring learning of employees in a formal mentoring program, if I were interested in development, I would conduct the quantitative survey first and then conduct the qualitative interviews with a sub-sample of employees who responded to my survey to probe deeper on the findings of the survey. That way, the results of my quantitative survey will help me to decide the questions and sample of my qualitative interviews. Similarly, I might want to conduct the qualitative interviews with a representative sample of employees first to understand the multiple perceptions of how the mentoring program has impacted their learning at work. Then, I can use the themes I identify from my qualitative data to construct and validate a scale on learning in mentoring relationships and use it in a follow-up quantitative survey with another group of employees participating in the program for purposes of validating the scale. In doing so, I have used the findings of one method to help inform the other method. For examples of sequential mixed method design studies, see Cho and Egan (2013) and Ivankova and Stick (2007).</p>


<p class="subhead_2">Emphasis of Approaches and MMR Purpose</p>
<p class="Para_No_Ind">Now, coming to the dimension of “emphasis of approaches”, how will a researcher decide if the quantitative and the qualitative strands should have equal emphasis or if one of the strands should have a dominant emphasis? If the researcher is interested in triangulation, initiation, or expansion, it would help to give equal weightage to both the quantitative and the qualitative strands. For instance, referring to my example above about a study exploring employees’ learning in a mentoring program, if I am interested in seeking convergence across the qualitative data from the interviews and the quantitative data from the survey (triangulation) or if I am interested to surface the inconsistencies between the two kinds of data (initiation), I would typically give equal priority to both as then I can argue for convergence or lack thereof between the two kinds of data in a meaningful manner. Likewise, if I am interested in expansion so that I am using the quantitative survey to answer one component of my inquiry (e.g., perceptions of learning as an outcome in the mentoring program) and the qualitative interviews to answer the other component of my inquiry (e.g., perceptions of the user-friendliness of the different features of the mentoring program and their impact on learning experiences in the program), then it makes sense to give equal priority to both strands as both strands are equally contributing towards expanding the breadth and range of my research study. For examples of mixed method studies giving equal weight to the quantitative and qualitative strands, see Taylor and Tashakkori (1997) and Scott and Sutton (2009).</p>
<p class="Para_Ind">On the contrary, if the researcher is interested in development, it would help to give higher weight to one of the strands. For instance, if I am using the qualitative interviews to elaborate or clarify the findings of my quantitative survey, it is evident that the quantitative survey conducted first is the major aspect of my study (carrying higher weightage) which then is followed by a small qualitative component to probe deeper some of the quantitative findings. Similarly, if I have conducted qualitative interviews first to gain a comprehensive understanding of the different perceptions of the ways in which the employees have experienced learning in the mentoring program so that the data from the interviews can help me to construct and validate a scale through a follow up survey, then it is typical to give higher weightage to the qualitative component as that has served as the major foundational aspect of my study. For examples of dominant status mixed method design studies, see Weaver-Hightower (2014) and Onwuegbuzie, Witcher, Collins, Filer, Wiedmaier, and Moore (2007).</p>
<p class="Para_Ind">As for complementarity, it would depend on how much critical the unique findings of one of the strands are over the other one in reference to the research question. For complementarity, while I am studying the same conceptual phenomenon of learning in mentoring relationships with the qualitative interviews and the quantitative survey, each method (i.e., qualitative or quantitative) can uniquely explore different aspects of the phenomenon. For instance, the in-depth approach of qualitative interview can help employees from minority backgrounds voice their frustration with challenges experienced in the program and the objective approach of the quantitative survey can help to identify the frequency of a certain kind of learning activity experienced by employees of different ethnic backgrounds. If my research question is primarily focused on understanding equity issues in mentoring programs from the participants’ perspectives, I would give higher weightage to the qualitative strand as the interview data can provide an in-depth understanding of the participants’ experiences. But, if my research question is primarily focused on deriving an overall understanding of discrepancies between the learning activities for employees belonging to different ethnical backgrounds in the program and participants’ lived experiences are of secondary concern, then, I would give higher weightage to the quantitative strand.</p>


<p class="subhead_2">Level of Mixing and MMR Purpose</p>
<p class="Para_No_Ind">In regards to the dimension of “level of mixing”, how will a researcher decide if the quantitative and the qualitative strands should be mixed only at the data interpretation stage (partially mixed) or across the other stages in research as well (fully mixed)? Now, if the researcher is interested in development, then mixing is needed in more than at the data interpretation stage. Referring to the example of the study exploring employees’ learning in a mentoring program, if I am interested to probe the quantitative findings of a survey through qualitative interviews, the quantitative findings will help me decide the questions I want to ask in my interviews and the sample of employees I want to interview. Similarly, if I am conducting in-depth interviews at first to gain an understanding of the phenomenon of learning in mentoring relationships with the objective of developing and validating a scale through a follow up quantitative study, the qualitative findings will help me to construct the scale which I will use in my quantitative survey. In both cases, I am then mixing qualitative and quantitative strands across the stages of research question, data collection, data analysis, and data interpretation.</p>
<p class="Para_Ind">As for the objectives of triangulation, complementarity, initiation, and expansion, the quantitative and the qualitative strands can be mixed either just at the data interpretation stage or they can be mixed across the stages of research objective, data collection, and data analysis. If I am using research objectives of both prediction and exploration and/or using the same instrument to collect both quantitative and qualitative data (e.g., survey with closed ended and open ended questions) and/or applying mixed data-analytic techniques, then I am mixing data more than just at the data interpretation stage and hence, employing a fully mixed design for my study. But, if I am collecting the quantitative and qualitative data separately and completing the data analysis of the two strands in its entirety before comparing and contrasting the findings at the data interpretation stage to see if the findings corroborate one another (triangulation), enhance one another (complementarity), contradict one another (initiation), and/or extend the range of my inquiry through addressing different components of my inquiry (expansion), then I am using partially mixed design for my study. For an example of fully mixed methods design study see Mazzola, Walker, Shockley, and Spector (2011) and for an example of partially mixed methods design study, see Senne and Rikard (2002).</p>



<p class="subhead_1">RESEARCH QUESTIONS IN MMR<a class="ThirdLevelTocAnchor" id="ach079sub5"></a></p>
<p class="Para_No_Ind">A strong mixed methods study needs to articulate research questions that include qualitative and quantitative components that are inter-connected (Tashakkori &amp; Creswell, 2007). That inter-connection can be achieved by ensuring that research questions are in sync with the stated purpose of a mixed methods study. The researcher can write separate questions for the quantitative and qualitative strands and then add an explicit mixed methods question that connects the quantitative and qualitative strand in a purposeful manner. Referring to the five purposes guiding mixed methods designs discussed above, the mixed method research questions can be drafted to fit those purposes (Greene et al., 1989). For instance, if I am interested in triangulation in my study about employees’ learning experiences in the mentoring program, I would include a question like: Do the quantitative and qualitative findings about learning in the mentoring program converge? If I am interested in complementarity, I would ask: How does the quantitative findings add to the qualitative findings about learning in the mentoring program or vice versa? For initiation, I would ask: Do the quantitative and qualitative findings about learning in the mentoring program contradict one another in any way? For development, I would include a question like: How do the follow-up interview findings help explain the survey findings about learning in the mentoring program? Only for expansion, I would not need to ask an explicit mixed methods question because the quantitative and qualitative strands are expected to focus on different parts of my inquiry (e.g., process and outcome) and hence the connection between them is obvious, i.e., the process focus of qualitative findings about the different features of the mentoring program and their impact on learning experiences in the program can help me understand the quantitative findings on whether learning is emerging as a statistically significant outcome of the mentoring program.</p>
<p class="Para_Ind">While advisable, not all mixed methods studies include an explicit mixed methods question. Instead, studies can include an overarching hybrid research question which is then broken into the quantitative and qualitative sub-questions. For my study on employees’ learning in the mentoring program, I can ask a hybrid question like: How is participation in a mentoring program linked to employee learning? And, then, my sub-questions can be: Do employees participating in the mentoring program learn more at work than those who do not?; How do employees experience learning in mentoring relationships initiated in the program? Needless to say that my former sub-question is meant for the quantitative strand and the later one is meant for the qualitative strand. Another option can be writing research questions for each phase of the study as the study evolves, but this option is only for studies that employ a sequential design.</p>


<p class="subhead_1">CONCLUSION<a class="ThirdLevelTocAnchor" id="ach079sub6"></a></p>
<p class="Para_No_Ind">This paper offered a brief introduction to MMR. First, I attempted to untangle the paradigmatic debate surrounding MMR and explain the relevance of a pragmatic approach for undertaking mixed method studies. Second, I illustrated how mixed methods designs can be mapped to the different purposes of conducting MMR. And, lastly, I presented an overview of different ways in which methods research questions can be presented. I hope that this basic introduction to MMR will encourage readers to delve deeper into the MMR literature and make informed decisions in designing and implementing MMR studies.</p>

<p class="Para_Quote">This research was previously published in the International Journal of Adult Vocational Education and Technology (IJAVET), 7(2); edited by Victor Wang, Judith E. Parker, and Geraldine Torrisi-Steele; pages 32-41, copyright year 2016 by IGI Publishing (an imprint of IGI Global).</p>




<p class="Ref_Hd">REFERENCES<a class="ThirdLevelTocAnchor" id="ach079sub7"></a></p><p class="Ref_Para">Campbell, D. T., &amp; Fiske, D. W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix.  <span class="Italic">Psychological Bulletin</span> , <span class="Italic">56</span>(2), 81–105. doi:10.1037/h0046016</p><p class="Ref_Para">Cho, Y., &amp; Egan, T. (2013). Organizational support for action learning in South Korean organizations.  <span class="Italic">Human Resource Development Quarterly</span> , <span class="Italic">24</span>(2), 185–213. doi:10.1002/hrdq.21154</p><p class="Ref_Para">Creswell, J. W., Plano Clark, V. L., Gutmann, M. L., &amp; Hanson, W. E. (2003). Advanced mixed methods research designs . In Tashakkori, A., &amp; Teddlie, C. (Eds.), <span class="Italic">Handbook of mixed methods in social and behavioral research</span>  (pp. 209–240). Thousand Oaks, CA: Sage Publications Inc.</p><p class="Ref_Para">Dewey, J., &amp; Boydston, J. A. (1925). <span class="Italic">John Dewey: The Later Works</span> . Carbondale, IL: Southern Illinois.</p><p class="Ref_Para">Eisenhardt, K. M. (1989). Building theories from case study research.  <span class="Italic">Academy of Management Review</span> , <span class="Italic">14</span>(4), 532–550.</p><p class="Ref_Para">Feilzer, M. Y. (2010). Doing mixed methods research pragmatically: Implications for the rediscovery of pragmatism as a research paradigm.  <span class="Italic">Journal of Mixed Methods Research</span> , <span class="Italic">4</span>(1), 6–16. doi:10.1177/1558689809349691</p><p class="Ref_Para">Feldon, D. F., Maher, M. A., Hurst, M., &amp; Timmerman, B. (2015). Faculty Mentors’, Graduate Students’, and Performance-Based Assessments of Students’ Research Skill Development.  <span class="Italic">American Educational Research Journal</span> , <span class="Italic">52</span>(2), 334–370. doi:10.3102/0002831214549449</p><p class="Ref_Para">Greene, J. C., Benjamin, L., &amp; Goodyear, L. (2001). The merits of mixing methods in evaluation.  <span class="Italic">Evaluation</span> , <span class="Italic">7</span>(1), 25–44. doi:10.1177/13563890122209504</p><p class="Ref_Para">Greene, J. C., &amp; Caracelli, V. J. (1997). <span class="Italic"><span class="Italics">Advances in mixed-method evaluation: the challenges and benefits of integrating diverse paradigms</span> (No. 658.4032 A244)</span> . San Francisco, CA: Jossey-Bass.</p><p class="Ref_Para">Greene, J. C., &amp; Caracelli, V. J. (2003). Making paradigmatic sense of mixed methods practice . In Tashakkori, A., &amp; Teddlie, C. (Eds.), <span class="Italic">Handbook of mixed methods in social and behavioral research</span>  (pp. 91–110). Thousand Oaks, CA: Sage Publications Inc.</p><p class="Ref_Para">Greene, J. C., Caracelli, V. J., &amp; Graham, W. F. (1989). Toward a conceptual framework for mixed-method evaluation designs.  <span class="Italic">Educational Evaluation and Policy Analysis</span> , <span class="Italic">11</span>(3), 255–274. doi:10.3102/01623737011003255</p><p class="Ref_Para">Guba, E. G., &amp; Lincoln, Y. S. (1988). Do inquiry paradigms imply inquiry methodologies?  In Fetterman, P. M. (Ed.), <span class="Italic">Qualitative Approaches to Evaluating Education: A Silent Scientific Revolution</span> . New York: Praeger.</p><p class="Ref_Para">Hoffman, B., &amp; Nadelson, L. (2010). Motivational engagement and video gaming: A mixed methods study.  <span class="Italic">Educational Technology Research and Development</span> , <span class="Italic">58</span>(3), 245–270. doi:10.1007/s11423-009-9134-9</p><p class="Ref_Para">Ivankova, N. V., &amp; Stick, S. L. (2007). Students’ persistence in a distributed doctoral program in educational leadership in higher education: A mixed methods study.  <span class="Italic">Research in Higher Education</span> , <span class="Italic">48</span>(1), 93–135. doi:10.1007/s11162-006-9025-4</p><p class="Ref_Para">Johnson, B., &amp; Turner, L. A. (2003). Data collection strategies in mixed methods research . In Tashakkori, A., &amp; Teddlie, C. (Eds.), <span class="Italic">Handbook of mixed methods in social and behavioral research</span>  (pp. 297–319). Thousand Oaks, CA: Sage Publications Inc.</p><p class="Ref_Para">Johnson, R. B., &amp; Onwuegbuzie, A. J. (2004). Mixed methods research: A research paradigm whose time has come.  <span class="Italic">Educational Researcher</span> , <span class="Italic">33</span>(7), 14–26. doi:10.3102/0013189X033007014</p><p class="Ref_Para">Johnson, R. B., Onwuegbuzie, A. J., &amp; Turner, L. A. (2007). Toward a definition of mixed methods research.  <span class="Italic">Journal of Mixed Methods Research</span> , <span class="Italic">1</span>(2), 112–133. doi:10.1177/1558689806298224</p><p class="Ref_Para">Leech, N. L., &amp; Onwuegbuzie, A. J. (2009). A typology of mixed methods research designs.  <span class="Italic">Quality &amp; Quantity</span> , <span class="Italic">43</span>(2), 265–275. doi:10.1007/s11135-007-9105-3</p><p class="Ref_Para">Maxwell, J. A., &amp; Loomis, D. M. (2003). Mixed methods design: An alternative approach. <span class="Italics">Handbook of mixed methods in social and behavioral research, 1</span>, 241-272.</p><p class="Ref_Para">Mazzola, J. J., Walker, E. J., Shockley, K. M., &amp; Spector, P. E. (2011). Examining stress in graduate assistants: Combining qualitative and quantitative survey methods.  <span class="Italic">Journal of Mixed Methods Research</span> , <span class="Italic">5</span>(3), 198–211. doi:10.1177/1558689811402086</p><p class="Ref_Para">McMillan, J. H., &amp; Schumacher, S. (2001). <span class="Italic">Research in education: A conceptual introduction</span>  (5th ed.). New York, NY: Longman.</p><p class="Ref_Para">Mintzberg, H. (1979). An emerging strategy of” direct” research.  <span class="Italic">Administrative Science Quarterly</span> , <span class="Italic">24</span>(4), 582–589. doi:10.2307/2392364</p><p class="Ref_Para">Morgan, D. L. (2007). Paradigms lost and pragmatism regained methodological implications of combining qualitative and quantitative methods.  <span class="Italic">Journal of Mixed Methods Research</span> , <span class="Italic">1</span>(1), 48–76. doi:10.1177/2345678906292462</p><p class="Ref_Para">Morse, J. M. (2003). Principles of mixed methods and multimethod research design . In Tashakkori, A., &amp; Teddlie, C. (Eds.), <span class="Italic">Handbook of mixed methods in social and behavioral research</span>  (pp. 189–208). Thousand Oaks, CA: Sage Publications Inc.</p><p class="Ref_Para">Onwuegbuzie, A. J., Johnson, R. B., &amp; Collins, K. M. (2009). Call for mixed analysis: A philosophical framework for combining qualitative and quantitative approaches. <span class="Italics">International journal of multiple research approaches, 3</span>(2), 114-139.</p><p class="Ref_Para">Onwuegbuzie, A. J., Witcher, A. E., Collins, K. M., Filer, J. D., Wiedmaier, C. D., &amp; Moore, C. W. (2007). Students’ perceptions of characteristics of effective college teachers: A validity study of a teaching evaluation form using a mixed-methods analysis.  <span class="Italic">American Educational Research Journal</span> , <span class="Italic">44</span>(1), 113–160. doi:10.3102/0002831206298169</p><p class="Ref_Para">Putnam, J. (1990). Realism with a human face . In Putnam, H. (Ed.), <span class="Italic">Realism with a human face</span>  (pp. 3–29). Cambridge, MA: Harvard University Press.</p><p class="Ref_Para">Scott, C., &amp; Sutton, R. E. (2009). Emotions and change during professional development for Teachers: A mixed methods study.  <span class="Italic">Journal of Mixed Methods Research</span> , <span class="Italic">3</span>(2), 151–171. doi:10.1177/1558689808325770</p><p class="Ref_Para">Senne, T. A., &amp; Rikard, G. L. (2002). Experiencing the portfolio process during the internship: A comparative analysis of two PETE portfolio models.  <span class="Italic">Journal of Teaching in Physical Education</span> , <span class="Italic">21</span>(3), 309–336.</p><p class="Ref_Para">Tashakkori, A., &amp; Creswell, J. W. (2007). Editorial: The new era of mixed methods.  <span class="Italic">Journal of Mixed Methods Research</span> , <span class="Italic">1</span>(1), 3–7. doi:10.1177/2345678906293042</p><p class="Ref_Para">Taylor, D. L., &amp; Tashakkori, A. (1997). Toward an understanding of teachers' desire for participation in decision making.  <span class="Italic">Journal of School Leadership</span> , <span class="Italic">7</span>(6), 609–628.</p><p class="Ref_Para">Teddlie, C., &amp; Tashakkori, A. (2003). Major issues and controversies in the use of mixed methods in the social and behavioral sciences . In Tashakkori, A., &amp; Teddlie, C. (Eds.), <span class="Italic">Handbook of mixed methods in social and behavioral research</span>  (pp. 3–50). Thousand Oaks, CA: Sage Publications Inc.</p><p class="Ref_Para">Teddlie, C., &amp; Tashakkori, A. (2006). A general typology of research designs featuring mixed methods.  <span class="Italic">Research in the Schools</span> , <span class="Italic">13</span>(1), 12–28.</p><p class="Ref_Para">Teddlie, C., &amp; Tashakkori, A. (Eds.). (2009). <span class="Italic">Foundations of mixed methods research: Integrating quantitative and qualitative approaches in the social and behavioral sciences</span> . Thousand Oaks, CA: Sage Publications Inc.</p><p class="Ref_Para">Weaver-Hightower, M. B. (2014). A mixed methods approach for identifying influence on public policy.  <span class="Italic">Journal of Mixed Methods Research</span> , <span class="Italic">8</span>(2), 115–138. doi:10.1177/1558689813490996</p></body></html>