CANDACE: I had not yet been introduced to qualitative software when I worked on my first qualitative research project, which involved participant observation and 10 formal interviews. The coding process was similar in terms of reviewing the data and assigning and organizing codes, but I did all of it by hand on hard copies of fieldnotes and transcripts using different colored highlighters to assign/indicate codes and paper folders for each coding category. Since that time, my research has evolved to larger projects and team research and I have used various iterations of QSR NVivo (previously NUD*IST). The software has not changed the underlying coding concepts and analysis, but facilitates collaboration and more sophisticated analysis than by hand in terms of allowing quick searches of simultaneous instances of multiple codes (e.g. caregiving, communication, and health decline) or searching on participant types (e.g., widowed women) or specific research sites.
TYLER: I have always used qualitative coding software for open coding, so unlike you, Candace, I can’t make a before-and-after comparison. I do think that using coding software allows me to be a bit more flexible in my approach to coding. I can easily recode a section or change the name of a code without going back through all of the previous transcripts and carrying out the change. This gives me the freedom to get it wrong the first time, which puts me at ease about coding. The other way that software changes the research process for me is in terms of writing. Having easy access to all of the quotes under one theme allows me to add lots of rich data examples into my manuscripts without returning to the raw data. This is an aspect of convenience that changes the narrative quality of my publications.
COLLEEN: Like you, Tyler, I have always used some sort of CAQDAS program, particularly for open coding. Over time as my knowledge and understanding of qualitative methods, and specifically formal coding, have increased I have gained a better sense of and greater comfort in how to use these tools in ways that work best for various research projects and me/my way of thinking and organizing. In particular, I have gained comfort in moving back and forth between paper and pencil strategies and using CAQDAS software. When possible I prefer to open code in ATLAS.ti, for reasons like the one you noted, Tyler, being able to change the names of codes as analyses evolve is really helpful. In addition, I have no counterfactual, but it seems my persistence with keeping a detailed codebook is likely better when using a CAQDAS program as it is so easy to update right as I am coding.
KATE: Colleen, your last point really resonates with me. I don’t think I would be nearly as good about keeping a detailed and up-to-date codebook if I had to do it in a separate document. Even being able to write code definitions in a CAQDAS program, I find that sometimes I neglect to update them as my thinking about the codes changes. I also appreciate how easy CAQDAS programs make it to pull together all of the quotes associated with one code. Having always used CAQDAS software, I can’t imagine having to do this by hand or using index cards laid out on a table or the floor. I think my analysis is more rigorous because I can easily pull together all of the quotes associated with one code and make sure they are all describing the same idea or phenomena. I do this multiple times and continue to do it as I write up my findings. I think I might be more likely to go with my first idea or understanding of the data if this were not so easy to do using a CAQDAS program.
TYLER: Colleen and Kate, I think your discussion about keeping a detailed codebook is a great one—and it relates to the issue of working on teams as well. When multiple people are creating codes, it is necessary for them to define them carefully so that others can come through and use the code as it was intended. CAQDAS programs make that process much easier.
KATE: Tyler, I have found the same thing, a CAQDAS program makes it easier to keep track of codes as a team. Though I find it’s also necessary, at least in the beginning, to talk in person or on the phone about the codes to make sure everyone is on the same page.
ANDREA: I agree with Kate and Tyler, that CAQDAS is useful with teams. I would add that the use of CAQDAS software programs is also helpful with large quantities of data. In examining differences between projects in which CAQDAS software was used versus not, I think the use related to the volume of data (but not depth of analysis) and the number of individuals involved in the analysis.
CANDACE: It is difficult for me to imagine how it would be possible to organize and manage large data sets collected and analyzed by multiple team members, including students, who may not be part of the study for the entire duration, without computer-assisted qualitative data analysis software. Everyone is able to work on the project simultaneously and frequently, remotely. The coder comparison feature allows us to understand how consistently coders are applying codes, which is essential in teamwork. And, multiple team members are also able to simultaneously enter and code data.
COLLEEN: Unlike Candace, it is working in larger teams that has somewhat precluded an extensive use of ATLAS.ti for me. Specifically, as I worked on a project with another faculty member (a Mac user) and a couple of graduate students a few years back, we discovered that despite the Mac version of ATLAS.ti being released, the Mac and PC versions didn’t talk well to each other. Specifically, we could not use the copy bundle feature across the Mac and PC versions of ATLAS.ti. My graduate student still checks in with IT support at ATLAS.ti every so often to understand if this feature has been fixed, and to my knowledge it still has not been fixed. As a result we used ATLAS.ti to store and manage the qualitative data for the entire project. For open coding, we were able to borrow a PC from our college for students to use, and conduct open coding in ATLAS.ti for the project this way. However, as we moved into axial coding our coding in teams moved back to paper. Candace, have you run into any issues using Macs and PCs with NVivo? Or have you had any issues during particular phases of formal coding?
CANDACE: The [in]compatibility issue between Mac and PC version of NVivo is one of the reasons that I and other faculty have resisted switching to Mac over the years. All of our team members use PCs and we have avoided any potential pitfalls. As we move to axial coding, we typically switch over to paper too and ultimately create analysis charts populated with data and diagrams in Microsoft Word. This move is not so much a software limitation, but a preference and possibly a user limitation that speaks to the next set of questions.
COLLEEN: That makes sense, Candace, especially in terms of user limitations—or project limitations. In some of the community-based participatory research (CBPR) work in which I am involved, as we work with community members to conduct coding, using a CAQDAS program becomes particularly cumbersome due to resources, as well as the flow of group coding meetings. We often open code using paper and pencil, and then as time consuming as it might be, we enter these open codes into ATLAS.ti to help with organizing codes for axial coding. We then conduct axial coding with community members by using analysis charts in Microsoft’s Excel program. As I reflect, it seems that my preference is to use CAQDAS software for open coding but depending on project circumstances this does not always look the same.
KATE: I have used Dedoose which is an online CAQDAS program for analysis with groups. It gets around the Mac/PC compatibility issues and allows for real time sharing. Everyone logs in to the same project and can see each other’s codes and coding. It’s also relatively inexpensive, which makes it easier to work with students. However, I think there are some limitations in terms of the interface and exporting codes to Word documents for work outside of the CAQDAS program.
TYLER: I think all of these comments speak to the challenges of conducting qualitative research in groups. Coding alone is tricky because it is easy to get entrenched in the data and lose perspective about the narrative you’re creating. However, working in groups poses practical challenges like computer compatibility and other struggles with how to get everyone on the same page during coding (literally and figuratively). I have worked with teams in Dedoose and I never felt we got to a good place with group coding. The limitations of the software (as Kate pointed out) paired with the lack of norms for coding in teams made meetings long and arduous. This seems to be an area where qualitative researchers might benefit from sharing resources about best practices for coding in groups. I can imagine that successful qualitative teams could articulate their processes in a way that might benefit others.
KATE: Yes! It would be great to hear more about how people do qualitative analysis in teams. This is something that I’m also working to figure out, especially for groups of three or more. It’s a shame that there usually isn’t room in journal articles to describe this process in detail.
ANDREA: CAQDAS software worked well for our team and the project goals. We had two laptops set up in the project laboratory that students would do coding on for the project using MAXQDA software. While this limited flexibility in terms of when and where students could code, this ensured that there were at least two people present that could discuss questions that emerged during the coding process.
CANDACE: I confess to not using NVivo to its fullest capacity and am working on learning more about the program, particularly its ability to facilitate analysis of network data. To date, I have mostly used the program to store, organize, and code the data. When it comes to analysis, the teams I have led or worked on mostly use node searches and queries to search on specific topics, participants, and research sites.
TYLER: Candace, my use of NVivo is nearly identical to you. I wonder sometimes if I am missing out on important features of the software, but it seems to work better for me to code and organize the data using NVivo and then step back and do the higher-order thinking through writing and modeling on paper. That said, I have recently started using MAXQDA to analyze a mixed methods dataset. Now that I have more quantitative data to go along with interview transcripts, it may be beneficial for me to learn about other features of this software to make the best use of the information I have.
COLLEEN: As I reflect on what both of you have written, as someone who also does not use ATLAS.ti to its fullest capacity, it makes me think of the time it takes to keep up with all the features of CAQDAS software programs, and how this bumps up against other time pressures of the academy and research-oriented positions. I think back to when I was a graduate student and what a luxury it was the attend a training on ATLAS.ti—I often reflect on the discussions in this training in which using ATLAS.ti with a team of coders was discussed (this was not my reality at the time as I was dissertating), and tell myself that I need to go back to my notes or explore this more. But because of resource constraints (time and money for training) I tend to stick with the features I know.
KATE: I am also guilty of not taking advantage of all of the features that ATLAS.ti and Dedoose have to offer, and primarily use them for open and axial coding. I am quite sure that there are features that I don’t even know exist but might find useful. Part of this is likely, as Colleen mentioned, the tradeoff between the time it takes to learn something new and just doing it the way you’ve always done it even if it might be less efficient. I wonder if part of it is also that for axial and selective coding some of this work and thinking happens when I’m away from my desk doing something else or talking to someone about the project and can then think through how to make sense of the data or am able to see the data in a new way. I think to some extent, for me, this work will never be done (or at least not exclusively done) while sitting in front of a computer using a CAQDAS program.
TYLER: Kate, I think you make a really important point about where, when, and how qualitative analysis really happens. When the researcher is the analytical tool, the work is happening all the time as we move through our day by having conversations with colleagues or chatting with our partners over dinner. For me, the most important analysis often happens while writing. I arrive at a tentative core concept or structure for the results and then I try to write it up with support from the data. I find out quickly where the gaps are and then I can return to the raw data (and the CAQDAS program) and regroup. So it’s also true for me that I will never do all of my analysis at the computer, even if I fully mastered a software package.
ANDREA: In a similar vein, I do not use MAXQDA to the fullest extent. I agree with Kate and Tyler on the importance of the researcher. For me, the software is a tool and extent of use depends on the aim of the study and the analytic strategy of the researcher. There are plenty of useful features to explore and that can help provide insight into data. In the study discussed in this book, the software was mainly used as a means to organize data and to easily retrieve segments of transcripts for the researchers to analyze. I have used different functions within MAXQDA to visualize codes, which has been useful to see codes in individual transcripts in relation to codes in other family member transcripts.
CANDACE: Aside from space, which is a very pragmatic issue, beyond mentioning use of a specific program, describing how it is used has not become part of the scientific reporting protocol. However, as qualitative software evolves, I think it would be very helpful to understand how researchers use it and under what circumstances.
TYLER: I agree with you, Candace, that we all tend to leave out details if we don’t think reviewers will be looking for them. Space is too precious. However, I think the other piece is that I don’t want to admit all of the ways I didn’t use the software. Candace said she had to confess that she doesn’t use NVivo fully, and I feel the same way. If I can avoid admitting that I only use qualitative software for organizing and basic coding, I do.
CANDACE: Well said, Tyler. If I think about it carefully, I believe that I share your underlying reason for not fully discussing how I use the software.
KATE: In addition to what the two of you have said, I am also very careful to say that I used ATLAS.ti (or whatever software program I used) to organize my data analysis because I want to be clear that the software did not do the analysis. I (or the team) did the analysis, the software just helped me keep track of it. I think how we use software for qualitative analysis is different from how it’s used for quantitative analysis and although the structure of the software and how it organizes and stores codes and memos may impact our analysis I think it has less of an impact than quantitative software. So I probably avoid talking in detail about the software I used so as to avoid giving the impression (mainly for audiences not familiar with qualitative analysis) that the software in some way produced the themes that I am reporting.
COLLEEN: I agree with everything stated by all of you. And I just looked back at a few publications to recall how extensively I noted the use of ATLAS.ti in my work. It seems I tend to indicate my use of ATLAS.ti for managing the data, but do not discuss how I used ATLAS.ti through each phase of analysis. Even more than space, I tend to reflect what Kate stated in terms of not wanting to give the impression that the CAQDAS program actually produced the themes. However, this conversation is making me think that by including more details regarding the use of CAQDAS programs, we may help educate audiences who are less familiar with qualitative analyses with exactly how the analyses are done—with CAQDAS software being but one (potential) tool in the analysis process.
ANDREA: I think space is the primary reason there is not as much discussion of CAQDAS. With limited space available in a manuscript, we need to communicate clearly and effectively our analytic process. While we used the software, we did the analytic process rather than the software doing it for us. For example, we had a reviewer state they were unclear on how we used the software and asked us to clarify our role in the analysis. This resulted in a focus on the process and our role and put the software in the background, which was true to the analytic process.