Christopher Kahn
Division of Emergency Medical Services and Disaster Medicine, Department of Emergency Medicine, University of California, San Diego, CA, USA
Congratulations on your desire to pursue research in emergency medical services! It is a field with many exciting research opportunities, but perhaps more than its share of challenges. As pointed out in a 2009 editorial of the same name as the above quote, “In few other areas of medicine are such profound differences in practice, equipment, and outcomes accepted or even observed.” [1] While the historical context that led to this is complex and interesting, at least if you are an emergency medical services (EMS) physician, the need for high-quality research to help provide a means for greater standardization and a decrease in regional disparities is great – and that is where you come in.
To make sure that your EMS research project is believable, you will need to address the following issues:
As indicated, there is tremendous variability in EMS systems. While there are certainly some national differences, such as prevalence of volunteer systems in some parts of the country and transport time differences in urban versus rural systems, the differences from one locality to the next can be pretty striking, too. If your system consists of an urban/suburban community served by a fire-based tiered EMS system that uses a combination of first responder fire units and paramedic ambulances, it might be hard to convince the medical director of a rural community served by a single paramedic ambulance and police first responders that your findings are applicable to his or her system. Even in systems that look fairly similar at first glance, there are often major differences in protocols and skills utilization that make it hard to compare one to the other. For example, there is a lot of variation in how paramedics approach prehospital airway management. Should you stay on scene to manage an airway instead of rapidly going to the closest emergency department? Should you use rapid sequence intubation medications? Should you intubate children? Should you intubate anybody?
Fortunately for you, you are not the first EMS researcher that has ever had to deal with this. Any person reading through the EMS research literature with a critical eye and an interest in seeing what might be of value to their system knows what their own system is like, and that absolves you of the need to find a way to make your results generalizable to every system (don’t worry, they won’t be). The most important step you can take to ensure that your research project has maximal external validity is to be specific in describing your system when you discuss your methodology. Make sure you detail items such as:
If you do not know this information, your local EMS medical director will.
Given the variability in EMS systems, many researchers assume it is too difficult to look at questions across more than one system. After all, it is important to make sure that your data are being collected in the same way, and that they are comparable across every system you are looking at. Unfortunately, while limiting your study to one system is undoubtedly easier – and a lot quicker if you are trying to get your study done in a hurry – you are going to be limited to whatever cases are present in your locality. For questions related to common problems such as minor trauma and some respiratory complaints, or for more general concerns such as response times and charting compliance, this may not be an issue. However, for questions related to some of the more critical but rarer concerns such as cardiac arrest, neonatal emergencies, and mass casualty incidents, you are going to spend a long time waiting for enough data to come through your system to make it worth reporting. To figure out whether this is the case, it is important to run a sample size calculation during the design phase of your study (Chapter 28).
One solution is to combine forces with other researchers. There are some large, government-funded multicenter research groups such as the Resuscitation Outcomes Consortium and the Neurological Emergencies Treatment Trials group. However, while these are great for people already in the network, privacy laws, grant requirements, and other concerns often make it difficult to approach these groups as an outsider and access their data. Instead, consider finding other people in your region that share an interest in the research question you have brought up and see if you can work together to get access to data from each of your systems (Chapter 26 has a discussion on figuring out authorship and division of work questions). Your local EMS medical director and (if you have a local academic program) EMS fellowship director or emergency medicine program director will likely have some idea of whom you might wish to contact. Not only will you have more data to power your study, you also just might create a network of colleagues that you can access again for future studies.
Another approach is to carry out a well-designed retrospective look at a reliable data set (Chapter 14). EMS is fortunate to have a national data set that is designed (in part) to be available for research questions: the National EMS Information System (NEMSIS). NEMSIS has its roots in the 1990s efforts by the National Highway Traffic Safety Administration (NHTSA) and other stakeholders to develop a uniform prehospital data set and allow national reporting of information. Although it was officially founded in 2001, it took another few years for NEMSIS to realize the vision of being a national repository for EMS information. Local agencies submit a subset of their data to their state, which, in turn, submits a smaller core subset to NEMSIS. The de-identified national data are available for on-line reporting at the NEMSIS web site.
Other data sets to consider looking at include your state’s EMS data repository and your state or local trauma registry.
Although other chapters in this book go into more detail on appropriate study design (such as Chapter 11), it is worth emphasizing that, given the other difficulties inherent in EMS research, you must ensure that your methodology is sound and uniformly applied. A study that is otherwise flawless will become essentially worthless if you cannot trust that the data are accurate and unbiased. For prospective studies, pay close attention to blinding, randomization, and the Hawthorne effect. For retrospective studies, use an appropriate guideline to ensure that data are retrieved, reviewed, and analyzed in a consistent fashion (Chapter 14).
For all EMS studies, it is a great idea to have somebody on your team who actually works in the practice setting you are studying. If you are looking at dispatch protocols, you need somebody to help you know what is going on in the dispatch world. Similarly, it is hard to accurately design a study that can be carried out successfully in the back of an ambulance if you do not have any prehospital field personnel helping you with the design. Not only will this add credibility to your results, it will make the entire process of designing the study, collecting the data, and discussing your results much richer.
Informed consent procedures are a wonderful use of research associate time (see Chapters 7, 8, and 16 on informed consent, and Chapter 25 on research associate programs). However, it is rarely practical to have research associates riding in the back of an ambulance with the prehospital team to consent patients; not only would you need an unwieldy number of associates to cover all units at all times of the day, but you would likely find that they got in the way of patient care. Further, unless your system only has one hospital, you are going to find that your research associates are generally not permitted to do anything in other hospitals.
This leaves the informed consent process as a trouble spot for EMS research. Although it usually is not a problem to obtain waivers of informed consent and Health Insurance Portability and Accountability Act (HIPAA) authorization for retrospective reviews, prospective studies will likely require a whole lot of work if they involve any kind of intervention or privacy risk and, therefore, are not exempt from review. Some of the informed consent methods that have worked for EMS researchers in the past include:
Speaking of patient outcomes, your study should focus on outcomes that people care about. Roughly 20 years ago, the NHTSA sponsored a workshop on EMS morbidity outcomes [2]. A list of high-value patient outcomes was developed, known as the “six Ds” (Box 15.1), that was described more comprehensively as part of the EMS Outcomes Project [3]. Your research will likely be regarded more highly if it focuses on outcome measures from this list.
Another way to help improve external validity is to choose a research question that focuses on issues of more general concern to the EMS community, rather than issues that are specific to the intricacies of your system. Although it is important to look at system-specific issues in the context of quality assurance and continuous quality improvement, you may find that this is less useful (or at least more frustrating) when it comes to peer review and scientific publication. There are several ways to get an idea of which topics might be more generalizable:
Alternatively, when you come across a research question that intrigues you, discreetly ask a few people in other EMS systems if it is an issue for them, as well. If none of your contacts in other systems have any concerns with the topic you have identified, it may be difficult to find peer reviewers that feel otherwise.
EMS research has a long and rich history of not being widely believed. A 1997 study reviewed 5842 MEDLINE publications focusing on prehospital care between 1985 and 1997, and found precisely one that was determined to have the ability to evaluate the efficacy of EMS. Fifty-four randomized controlled trials were found, but only seven showed an uncontradicted positive outcome, and only one of those focused on a major outcome such as survival [4]. There has certainly been progress since that rather dismal showing, but even today there is often a perception that EMS research is not of sufficient quality to inform our clinical practice. Your EMS research will hopefully avoid this concern by addressing the points mentioned in this chapter.