By now you should have an understanding of the various types of health care information systems and the value they can bring to health care organizations and the patients they serve. This chapter describes the typical process a health care organization goes through in acquiring or selecting a new clinical or administrative application. Acquiring an information system (IS) application can be an enormous investment for health care organizations. Besides the initial cost, there are a host of long-term costs associated with maintaining, supporting, and enhancing the system. Health care professionals need access to reliable, complete, and accurate information in order to provide effective and efficient health care services and to achieve the strategic goals of the organization. Selecting the right application, one that meets the organization’s needs, is a critical step. Too often information systems are acquired without exploring all options, without evaluating costs and benefits, and without gaining sufficient input from key constituent user groups. The results can be disastrous.
This chapter describes the people who should be involved, the activities that should occur, and the questions that should be addressed in acquiring any new information system. The suggested methods are based on the authors’ years of experience and on countless case studies of system acquisition successes and failures published in the health care literature.
In this book system acquisition refers to the process that occurs from the time the decision is made to select a new system (or replace an existing system) until the time a contract has been negotiated and signed. System implementation is a separate process described in the next chapter, but both are part of the systems development life cycle. The actual system selection, or acquisition, process can take anywhere from a few days to a couple of years, depending on the organization’s size, structure, complexity, and needs. Factors such as whether the system is deemed a priority and whether adequate resources (time, people, and funds) are available can also directly affect the time and methods used to acquire a new system (Jones et al., 2011).
Prior to arriving at the decision to select a new system, the health care executive team should engage in a strategic IS planning process, in which the strategic goals of the organization are formulated and the ways in which information technology (IT) will be employed to aid the organization in achieving its strategic goals and objectives are discussed. We discuss the need for aligning the IT plans with the strategic goals of the organization and for determining IT priorities in Chapter Thirteen. In this chapter, we assume that a strategic IT plan exists, IT priorities have been established, the new system has been adequately budgeted, and the organization is ready to move forward with the selection process. We also assume that the organization has conducted a readiness assessment and is well equipped to move forward with the health IT project or initiative. The AHRQ National Resource Center for Health IT has publicly available a number of tools that can be helpful to health care organizations in assessing their readiness health IT projects such as EHR implementations and for ensuring that they have in place the personnel, technical, and financial resources to embark on the initiative. These tools can be found at healthit.ahrq.gov/Survey_Compendium.
No board of directors would recommend building a new health care facility without an architect’s blueprint and a comprehensive assessment of the organization’s and the community’s needs and resources. The architect’s blueprint helps ensure that the new facility has a strong foundation, is well designed, fosters the provision of high-quality care, and has the potential for growth and expansion. Similarly, the health care organization needs a blueprint to aid in the planning, selection, implementation, and support of a new health care information system. The decision to invest in a health care information system should be well aligned with the organization’s overall strategic goals and should be made after careful thought and deliberation. Information systems are an investment in the organization’s infrastructure, not a one-time purchase. Health care information systems require not only up-front costs and resources but also ongoing maintenance, support, upgrades, and eventually, replacement.
The process an organization generally goes through in planning, selecting, implementing, and evaluating a health care information system is known as the systems development life cycle (SDLC). Although the SDLC is most commonly described in the context of software development, the process also applies when systems are purchased from a vendor or leased through an application service provider (ASP). An ASP is a company that deploys, hosts, and manages one or more clinical or administrative information systems through centrally located servers (generally via the Internet), often on a fixed, per-use basis or on a subscription basis. Regardless of how the system is acquired, most health care organizations follow a structured process for selecting and implementing a new computer-based system. The systems development process itself involves participation from individuals with different backgrounds and areas of expertise. The specific mix of individuals depends on the nature and scope of the new system.
Many SDLC frameworks exist, some of which employ an incremental approach, but most have four general phases, or stages—planning and analysis, design, implementation, and support and evaluation (Wager & Lee, 2006) (see Figure 7.1). Each phase has a number of tasks that need to be performed. In this chapter we focus on the first two phases; Chapter Eight focuses on the last two.
Figure 7.1. Systems development life cycle
The SDLC approach assumes that this four-phase life of an IS starts with a need and ends when the benefits of the system no longer outweigh its maintenance costs, at which point the life of a new system begins (Oz, 2012). Hence, the entire project is called a life cycle. After the decision has been made to explore further the need for a new information system, the feasibility of the system is assessed and the scope of the project defined (in actuality it is at times difficult to tell when this decision making ends and analysis begins). The primary focus of this planning and analysis phase is on the business problem, or the organization’s strategy, independent of any technology that can or will be used. During this phase, it is important to examine current systems and problems in order to identify opportunities for improvement. The organization should assess the feasibility of the new system—is it technologically, financially, and operationally feasible? Furthermore, sometimes it is easy to think that implementing a new IS will solve all information management problems. Rarely, if ever, is this the case. But by critically evaluating existing systems and workflow processes, the health care team might find that current problems are rooted in ineffective procedures or lack of sufficient training. Not always is a new system needed or the answer to a problem.
Once it is clear that a new IS is needed, the next step is to assess the information needs of users and define the functional requirements: What functions must the system have to fulfill the need? This process can be very time consuming. However, it is vital to solicit widespread participation from end users during this early stage—to solicit and achieve buy-in. As part of the needs assessment, it is also helpful to gather, organize, and evaluate information about the environment in which the new system is to operate. Through defining system requirements, the organization specifies what the system should be able to do and the means by which it will fulfill its stated goals.
Once the team knows what the organization needs, it enters the second stage, the design phase, where it considers all its options. Will the new system be designed in-house? Will the organization contract with an outside developer? Or will the organization purchase a system from a health information systems vendor or contract with an ASP? A large majority of health care organizations purchase a system from a vendor or at least look first at the systems available on the market. System design is the evaluation of alternative solutions to address the business problem. It is generally in this phase that all alternatives are considered, a cost-benefit analysis is done, a system is selected, and vendor negotiations are finalized.
After the contract has been finalized or the system has been chosen, the third phase, implementation, begins. The implementation phase requires significant allocation of resources in completing tasks such as conducting workflow and process analyses, installing the new system, testing the system, training staff, converting data, and preparing the organization and staff for the go-live of the new system. Finally, once the system is put into operation, the support and evaluation phase begins. It is common to underestimate the staff and resources needed to effectively keep new and existing information systems functioning properly. No matter how much time and energy were spent on the design and build of the application, you can count on the fact that changes will need to be made, glitches fixed, and upgrades installed. Likewise, most mission-critical systems need to be functioning 99.99 percent of the time—that is, with little downtime. Sufficient resources (people, technology, infrastructure, and upgrades) need to be allocated to maintain and support the new system. Although support costs can vary widely from system to system, in most industries up to 80 percent of the IS budget is spent on maintenance (Oz, 2012). The major reason for this significant proportion is that support is the longest phase in a system’s life cycle.
Moreover, maintaining and supporting the new system is not enough. Health care executives and boards want to know the value of the IT investment, thus the degree to which the new system has achieved its goals and objectives should be assessed. Eventually, the system will be replaced and the SDLC process begins again.
With this general explanation of the SDLC established, we begin by focusing on the first two phases—the planning and analysis phase and the design phase. Together they constitute what we refer to as the system acquisition process.
To gain an understanding of and appreciation for the activities that occur during the system acquisition process, we will follow a health care facility through the selection process for a new information system—specifically, an electronic health record (EHR) system. In this case the organization, which we will call Valley Practice, is a multiphysician primary care practice.
What process should the practice use to select the EHR? Should it purchase a system from a vendor, contract with an application service provider, or seek the assistance of a system developer? Who should lead the effort? Who should be involved in the process? What EHR products are available on the market? How reputable are the vendors who develop these products? These are just a few of the many questions that should be asked in selecting a new IS.
Although the time and the resources needed to select an EHR (or any health care information system) may vary considerably from one setting to another, some fundamental issues should be addressed in any system acquisition initiative. The sections that follow the case study describe in more detail the major activities that should occur (Exhibit 7.1), relating them to the multiphysician practice scenario. We assume that the practice wishes to purchase (rather than develop) an EHR system. However, we briefly describe other options and point out how the process may differ when the EHR acquisition process occurs in a larger health care setting, such as a hospital.
One of the first steps in any major project such as an EHR acquisition effort is to create a project steering committee. This committee’s primary function is to plan, organize, coordinate, and manage all aspects of the acquisition process. Appointing a project manager with strong communication skills, organizational skills, and leadership abilities is critical to the project. In our Valley Practice case, the project manager was a physician partner. In larger health care organizations such as hospitals, where a CIO is employed, the CIO would likely be involved in the effort and might also be asked to lead it.
Increasingly, clinicians such as physicians and nurses with training in informatics are being called on to lead clinical system acquisition and implementation projects. Known as chief medical informatics officers or nursing informatics officers, these individuals bring to the project a clinical perspective as well as an understanding of IT and information management processes. Regardless of the discipline or background of the project manager (for example, IT, clinical, or administrative), he or she should bring to the project passion, interest, time, strong interpersonal and communication skills, and project management skills and should be someone who is well respected by the organization’s leadership team and who has the political clout to lead the effort effectively.
Pulling together a strong team of individuals to serve on the project steering committee is also important. These individuals should include representatives from key constituent groups in the practice. At Valley Practice, a physician partner, a nurse, the business officer manager, and the CEO agreed to serve on the committee, along with a representative from the Regional Extension Center. Gaining project buy-in from the various user groups should begin early. This is a key reason for inviting representatives from key constituent groups to serve on the project steering committee. They should be individuals who will use the EHR system directly or whose jobs will be affected by it.
Consideration should also be given to the size of the committee; typically, having five to six members is ideal. In a large facility, however, this may not be possible. The committee for a hospital might have fifteen to twenty members, with representatives from key clinical areas such as laboratory medicine, pharmacy, and radiology in addition to representatives from the administrative, IT, nursing, and medical staffs.
It is important to have someone knowledgeable about IT serving on the project steering committee. This may be a physician, a nurse, the CEO, or an outside consultant. In a physician group practice such as Valley Practice, having an in-house IT professional is rare. The committee chair might look internally to see if someone has the requisite IT knowledge, skills, and interests and also the time to devote to the project, but also might look externally for a health care IT professional who might serve in a consultative role and help the committee direct its activities appropriately. In this case, the REC specialist may be able to provide both the technical and system selection expertise to assist Valley Practice in the acquisition process.
Once the project steering committee has been established, its first order of business is to clarify the charge to the committee and to define project goals. The charge describes the scope and nature of the committee’s activities. The charge usually comes from senior leadership or a lead physician in the practice. Project goals should also be established and communicated in well-defined, measurable terms. What does the committee expect to achieve? What process will be used to ensure the committee’s success? How will milestones be acknowledged? How will the committee communicate progress and resolve problems? What resources (such as time, personnel, and travel expenses) will the committee need to carry out its charge? What method will be used to evaluate system options? Will the committee consider contracting with a system developer to build a system or outsourcing the system to an application service provider? Or is the committee only considering systems available for purchase from a health care information systems vendor?
Once project goals are formulated, they can guide the committee’s activities and also clarify the resources needed and the likely completion date for the project. Here are some examples of typical project goals:
As part of the goal-setting process, the committee should determine the extent to which various options will be explored. For example, the Valley Practice project steering committee decided at the onset that it was going to consider only EHR products available in the vendor community and certified by the ONC EHR. As discussed in Chapter Six, to achieve meaningful use, health care provider organizations must use certified EHR products. A listing of certified projects are available on the Office of the National Coordinator for Health Information Technology web site. Users can be assured that certified EHR products meet certain standards for content, functionality, and interoperability.
The committee further stipulated that it would consider only vendors with experience (for example, five or more years in the industry) and those with a solid track record of system installations (for example, twenty-five or more installations). The committee members felt the practice should contract with a system developer only if they were unable to find a suitable product from the vendor community—their rationale being that the practice wanted to be known as high-tech, high-touch. They also believed it was important to invest in IT personnel who could customize the application to meet practice needs and who would be able to assist the practice in achieving project and practice goals.
Concurrently with the establishment of project goals, the project steering committee should conduct its first, cursory review of the EHR marketplace and begin investigating vendor profiles. Many resources are available to aid the committee in this effort. For example, the Valley Practice committee might obtain copies of recent market analysis reports—from research firms such as Gartner or KLAS—listing and describing the vendors that provide EHR systems for ambulatory care facilities. The committee might also attend trade shows at conferences of professional associations such as the Healthcare Information and Management Systems Society (HIMSS) and the American Medical Informatics Association (AMIA). (Appendix A provides an overview of the health care IT industry and describes a variety of resources available to health care organizations interested in learning about health care IT products, such as EHR systems, available in the vendor community.)
Besides identifying project goals, the project steering committee should define system goals. System goals can be derived by answering questions such as: What does the organization hope to accomplish by implementing an EHR system? What is it looking for in a system? If the organization intends to transform existing care processes, can the system support the new processes? Such goals often emerge during the initial strategic planning process when the decision is made to move forward with the selection of the new system. At this point, however, the committee should state its goals and needs for a new EHR system in clearly defined, specific, and measurable terms. For example, a system goal such as “select a new EHR system” is very broad and not specific. Here are some examples of specific and measurable goals for a physician practice.
Our EHR system should
These are just a few of the types of system goals the project steering committee might establish as it investigates a new EHR for the organization. The system goals should be aligned with the strategic goals of the organization and should serve as measures of success throughout the system acquisition process.
Once the goals of the new system have been established, the project steering committee should begin to determine system requirements. These requirements may address everything from what information should be available to the provider at the point of care to how the information will be secured to what type of response time is expected. The committee may use any of a variety of ways to identify system requirements. One approach is to have a subgroup of the committee conduct focus-group sessions or small-group interviews with the various user groups (physicians, nurses, billing personnel, and support staff). A second approach is to develop and administer a written or an electronic survey, customized for each user group, asking individuals to identify their information needs in light of their job role or function. A third is to assign a representative from each specific area to obtain input from users in that area. For example, the nurse on the Valley Practice project steering committee might interview the other nurses; the business office manager might interview the support staff. System requirements may also emerge as the committee examines templates provided by consultants or peer institutions, looks at vendor demonstrations and sales material, or considers new regulatory requirements the organization must meet.
The committee may also use a combination of these or other approaches. At times, however, users do not know what they want or will need. Hence, it can be extremely helpful to hold product demonstrations, meet with consultants, or visit sites already using EHR systems, so that those who will use or be affected by the EHR can see and hear what is possible. Whatever methods are chosen to seek users’ information system needs, the end result should be a list of requirements and specifications that can be prioritized or ranked. This ranking should directly reflect the specific strategic goals and circumstances of the organization.
The system requirements and priorities will eventually be shared with vendors or the system developer; therefore, it is important that they be clearly defined and presented in an organized, easy-to-understand format. For example, it may be helpful to organize the requirements into categories such as software (system functionality, software upgrades), technical infrastructure (hardware requirements, network specifications, backup, disaster recovery, security), and training and support (initial and ongoing training, technical support). These requirements will eventually become a major component of the request for proposal (RFP) submitted to vendors or other third parties (discussed next).
Once the organization has defined its system requirements, the next step in the acquisition process is to package these requirements into a structure that a third party can respond to, whether that third party be a development partner or a health information systems vendor. Many health care organizations package the requirements into a request for proposal (RFP). The RFP provides the vendor with a comprehensive list of system requirements, features, and functions and asks the vendor to indicate whether its product or service meets each need. Vendors responding to an RFP are also generally required to submit a detailed and binding price quotation for the applications and services being sought.
RFPs tend to be highly detailed and are therefore time consuming and costly to develop and complete. However, they provide the health care organization and each vendor with a comprehensive view of the system needed. Health care IT consultants can be extremely resourceful in assisting the organization with developing and packaging the RFP. An RFP for a major health care information system acquisition generally contains the following information (sections marked with an asterisk [*] are completed by the vendor; the other sections are completed by the organization issuing the RFP):
The RFP may become the basis for a legally binding contract or obligation between the vendor and the solicitor, so it is important for both parties to carefully consider the wording of questions and the corresponding responses (AHIMA, 2007).
RFPs are not the only means by which to solicit information from vendors. A second approach that is often used is the request for information (RFI). An RFI is less formal, considerably shorter than an RFP, and less time consuming to develop. It is often used as part of the fact-finding process to obtain basic information on the vendor’s background, product descriptions, and service capabilities. Some health care organizations send out an RFI before distributing the RFP, in order to screen out vendors whose products or services are not consistent with the organization’s needs or to narrow the field of vendors to a manageable number. The RFI can serve as a tool in gathering background information on vendors’ products and services and in providing the project steering committee with a better sense of the health IT market place. How does one decide whether to use an RFP, an RFI, both, or neither during the system acquisition process? Several factors should be considered. Although time consuming to develop, the RFP is useful in forcing a health care organization to define its system goals and requirements and prioritize its needs. The RFP also creates a structure for objectively evaluating vendor responses and provides a record of documentation throughout the acquisition process. System acquisition can be a highly political process; by using an RFP the organization can introduce a higher degree of objectivity into that process. RFPs are also useful data collection tools when the technology being selected is established and fully developed, when there is little variability between vendor products and services, when the organization has the time to fully evaluate all options, and when the organization needs strong contract protection from the selected vendor (DeLuca & Enmark, 2002). However, not all vendors may wish to submit a response to an RFI or RFP due to costs or suitability.
There are also drawbacks to RFPs. Besides taking considerable time to develop and review, they can become cumbersome, so detail oriented that they lose their effectiveness. For instance, it is not unusual to receive three binders full of product and service information from one vendor. If ten vendors respond to an RFP (about five is ideal), the project steering committee may be overwhelmed and find it difficult to wade through and differentiate among vendor responses. Having too much information to summarize can be as crippling to a committee in its deliberations as having too little.
Therefore a scaled-back RFP or an RFI might be a desirable alternative. An RFI might be used when the health care organization is considering only a small group of vendors or products or when it is still in the exploratory stages and has not yet established its requirements. Some facilities use an even less formal process consisting primarily of site visits and system demonstrations.
Regardless of the tool(s) used, it is important for the health care organization to provide sufficient detail about its current structure, strategic IT goals, and future plans that the vendor can respond appropriately to its needs. Additionally, the RFP or RFI (or variation of either) should result in enough specific detail that the organization gets a good sense of the vendor—its services, history, vision, stability in the marketplace, and system or product functionality. The organization should be able to easily screen out vendors whose products are undeveloped or not yet fully tested (DeLuca & Enmark, 2002).
In our Valley Practice case, the physicians and staff opted to acquire an EHR system from the vendor community. Organizations like Valley Practice often turn to the market for products that they will run on their own IT infrastructure. But there are times when they do not go to the market—they choose to leverage someone else’s infrastructure (by contracting with an application service provider) or they build the application (by contracting with a system developer or using in-house staff).
In recent years, with wider availability of high-speed or broadband Internet connections, more sophisticated vendor solutions, and a growing number of options for hosting software, the application service provider (ASP) approach has emerged as an alternative to buying, installing, and maintaining information systems. These services may be offered via “cloud” computing. Cloud computing is a general term that refers to both the applications delivered as services over the Internet and the hardware and software in the data centers that provide those services. An ASP is an organization with whom health care providers contract on a subscription basis to deliver an application and provide the associated services to support it. It’s somewhat analogous to the option of leasing rather than purchasing a car. ASPs are also similar in concept to the shared-systems option used by many hospitals in the 1960s and 1970s, when they could not afford or did not have the IT staff available in-house to run and support software applications and hardware. In essence, another organization houses and maintains the clinical or administrative application and related hardware; the health care organization or provider simply accesses the system remotely over a network connection and pays the monthly or negotiated fees. It is worth noting that some ASPs do not physically host the application but instead contract with a third-party data center to do so (Fortin & MacDonald, 2006). Health information exchange organizations could serve as ASPs as they mature and expand their capabilities.
Why might a health care organization consider contracting with an ASP rather than purchasing an EHR system (or other application) from a vendor? There are several reasons. First, the facility may not have the IT staff needed to run or support the desired system. Hiring qualified personnel at the salaries they demand may be difficult, and retaining them may be equally challenging. Second, ASPs typically enable health care organizations to use clinical or administrative applications with fewer up-front costs and less capital. For a small physician practice, these financial arrangements can be particularly appealing. Because ASPs offer fixed monthly fees or fees based on usage, organizations are better able to predict costs. Third, by contracting with an ASP, the health care organization can focus on its core business and not get bogged down in IT support issues, although it may still have to deal with issues of system enhancements, user needs, and the selection of new systems. Other advantages to using ASPs are rapid deployment and 24/7 technical support. They also offer scalability and flexibility, so as the practice or organization grows or shrinks in size or volume, they pay only for the services used.
ASPs also have some disadvantages and limitations that the health care organization should consider in its deliberations. Although rapid deployment of the application can be a tremendous advantage to an organization, the downside is the fact that the application will likely be a standard, off-the-shelf product, with little if any customization. This means that the organization has to adapt or mold its operations to the application rather than tailoring the application to meet the operational needs of the organization. A second drawback deals with technical support. Although technical support is generally available from an ASP, it is unrealistic to think that the ASP’s support personnel will have intimate knowledge of the organization and its operations. Frustrations can mount when one lacks in-house IT technical staff when and where they are needed. Third, health care providers have long been concerned about data ownership, security, and privacy—worries that increase when another organization hosts their clinical data and applications. How the ASP will secure data and maintain patient privacy should be clearly specified in the contract. Likewise, to minimize downtime, the ASP should have clear plans for backing up data, preventing disasters, and recovering data.
First Consulting prepared a report for the California HealthCare Foundation outlining developments, benefits, challenges, issues, and concerns related to the ASP model for ambulatory clinical applications (Fortin & MacDonald, 2006). It suggests health care leaders ask themselves four important questions in deciding whether an ASP approach is the right choice for their organization:
As the industry matures, we will likely see different variations and greater choices among organizations serving as ASPs. The health care executive considering whether an ASP is the right choice should thoroughly research the company and its products and consider factors such as company viability, target market, functionality, integration, implementation and training, help desk support, security, pricing, and service levels. It is important to be able to trust the ASP and to choose one wisely.
A second alternative to purchasing a system from a vendor is to contract with a developer to design a system for your organization. The developer may be employed in-house or by an outside firm. Working with a system developer can be a good option when the health care organization’s needs are highly uncertain or unique and the products available on the market do not adequately meet these needs. Developing a new or innovative application can also give the organization a significant competitive advantage. The costs and time needed to develop the application can be significant, however. It is also important to consider the long-term costs. Should the developer leave, how difficult would it be to hire and retain someone to support and maintain the system? How will problems with the system be addressed? How will the application be upgraded? What long-term value will it bring the organization? These are a few of the many questions that should be addressed in considering this option. It is rare for a health care organization to develop its own major clinical information system.
In the Valley Practice case, the project steering committee decided to focus its efforts at first on considering only EHR products available for purchase in the vendor community. The committee came to this conclusion after its initial review of the EHR marketplace. Committee members felt there were a number of vendors whose products appeared to meet practice needs. They also felt strongly that in-house control of the EHR system was important to achieving the practice goal of becoming a high-tech, high-touch organization, because they wanted to be able to customize the application. Realizing this, the committee had budgeted for an IT director and an IT support staff member. Members felt that the long-term cost savings from implementing an EHR would justify these two new positions.
The project steering committee at Valley Practice decided to go through the RFP process. It developed criteria by which it would review and evaluate vendor proposals. Criteria were used to grade each vendor’s response to the RFP. Grading scales were established so the committee could accurately compare vendors’ responses. These grading scales involved assigning more weight to required items and less weight to those deemed merely desirable. Categories of “does not meet requirement,” “partially meets requirement,” and “meets requirement” were also used. RFP documents were compared item by item and side by side, using the grading scales established by the committee (see Table 7.1 for sample criteria). To avoid information overload, a common condition in the RFP review process, the project steering committee focused on direct responses to requirements and referred to supplemental information only as needed. Summary reports of each vendor’s response to the RFP were then prepared by a small group of committee members and distributed to the committee at large.
Table 7.1. Sample criteria for evaluation of RFP responses
During the vendor review process, it is important to host vendor system demonstrations. The purpose of these demonstrations is to give the members of the health care organization an opportunity to (1) evaluate the look and feel of the system from a user’s point of view, (2) validate how much the vendor can deliver of what has been proposed, (3) conduct system usability testing, and (4) narrow the field of potential vendors. It is often a good idea to develop demonstration scripts and require all vendors to present their systems in accordance with these scripts. Scripts generally reflect the requirements outlined in the RFP and contain a moderate level of detail. For example, a script might require demonstrating the process of registering a patient or renewing a prescription. The use of scripts can ensure that all vendors are evaluated on the same basis or functionality. At the same time, it is important to allow vendors some creativity in presenting their product and services. When scripts are used, they need to be provided to vendors at least one month in advance of the demonstration, and both vendors and health care organization must adhere to them. It is also important to have end-users carry out certain functions or procedures that they would normally do in the course of the day using the vendor’s system. You might ask them to complete a system usability survey after they have had a chance to use the system and practice on several records. Figure 7.2 is an example of a system usability scale questionnaire in which end-users are asked to respond to each item using a Likert scale of 1 to 5, from strongly disagree to strongly agree.
Figure 7.2. System usability scale questionnaire
Source: Brooke, 1996; Lewis & Sauro, 2009.
Criteria should be developed and used in evaluating vendor demonstrations, just as they are for reviewing vendor responses to the RFP.
After reviewing the vendors’ RFPs and evaluating their product demonstrations, it is advisable to make site visits and check references. By visiting other facilities that use a vendor’s products, the health care organization should gain additional insight into what the vendor would be like as a potential partner. It can be extremely beneficial to visit organizations similar to yours. For instance, in the Valley Practice case, representatives from key practice constituencies decided to visit other ambulatory care practices to see how a specific system was being used, the problems that had been encountered, and how these problems had been addressed. The Regional Extension Center specialist also provided useful information to them.
How satisfied are the staff with the system? How responsive has the vendor been to problems? How quickly have problems been resolved? To what degree has the vendor delivered on its promises? Hearing answers to such questions firsthand from a variety of users can be extremely helpful in the vendor review process.
A host of other strategies can be used to evaluate a vendor’s reputation and product and service quality. Organizational representatives might attend vendor user-group conferences, review the latest market reports, consult with colleagues in the field, seek advice from consultants, and request an extensive list of system users.
Throughout the vendor review process, the project steering committee members should have evaluation tools in place to document their impressions and the views of others in the organization who participate in any or all of the review activities (review of RFPs, system demonstrations, site visits, reference checks, and so forth). The committee should then prepare vendor analysis reports that summarize the major findings from each of the review activities. How do the vendors compare in reputation? In quality of their product? In quality of service? How do the systems compare in terms of their initial and ongoing costs? To what degree is the vendor’s vision for product development aligned with the organization’s strategic IT goals?
The final analysis should include an evaluation of the cost and benefits of each proposed system. Figure 7.3 shows a comparison of six vendor products. Criteria were developed to score and rank each vendor’s system. As the figure illustrates, the selection committee ranked vendor 4 the top choice.
Figure 7.3. Cost-benefit analysis
Source: Partners HealthCare.
The capital cost analysis may include software, hardware, network or infrastructure, third-party, and internal capital costs. The total cost of ownership should factor in support costs and the costs of the resources needed (including personnel) to implement and support the system. Once the initial and ongoing costs are identified, it is important to weigh them against the benefits of the systems being considered. Can the benefits be quantified? Should they be included in the final analysis?
Assuming the capital cost analysis supports the organization in moving forward with the project, the project steering committee should compile a final report that summarizes the process and results from each major activity or event. The report may include
It is generally advisable to have two or three vendors in the final ranking, in the event that problems arise with the first choice during contract negotiations, the final step in the system acquisition process.
The final step of the system acquisition process is to negotiate a contract with the vendor. This, too, can be a time-consuming process, and therefore it is helpful to seek expert advice from business or legal advisors. The contract outlines expectations and performance requirements, who is responsible for what (for example, training, interfaces, support), when the product is to be delivered (and vendor financial liability for failing to deliver on time), how much customization can be performed by the organization purchasing the system, how confidentiality of patient information will be handled, and when payment is due. The devil is in the details, and although most technical terms are common between vendors, other language and nuances are not. Establish a schedule and a preimplementation plan that includes a timeline for implementation of the applications and an understanding of the resource requirements for all aspects of the implementation, including cultural change management, workflow redesign, application implementation, integration requirements, and infrastructure development and upgrades, all of which can consume substantial resources.
Throughout the course of the system acquisition project, a lot of materials will be generated, many of which should be maintained in a project repository. A project repository serves as a record of the project steering committee’s progress and activities. It includes such information and documents as minutes of meetings, correspondence with vendors, the request for proposal or request for information, evaluation forms, and summary reports. This repository can be extremely useful when there are changes in staff or in the composition of the committee and when the organization is planning for future projects. The project manager should assume a leadership role in ensuring that the project repository is established and maintained. Here is a sample of the typical contents of a project repository.
Managing the various aspects of the project and coordinating activities can be a challenging task, particularly in large organizations or when a lot of people are involved and many activities are occurring simultaneously. It is important that the project manager helps those involved to establish clear roles and responsibilities for individual committee members, to set target dates, and to agree upon methods for communicating progress and problems. Many project management tools exist that can be useful here. For example, a simple Gantt chart (Figure 7.4) can document project objectives, tasks and activities, responsible parties, and target dates and milestones. A Gantt chart can also display a graphical representation of all project tasks and activities, showing which ones may occur simultaneously and which ones must be completed before another task can begin. Other tools enable one to allocate time, staff, and financial resources to each activity. (Gantt charts and other timelines can be created with software programs such as Visio or Microsoft Project. A discussion of these tools is beyond the scope of this book, but can be found in most introductory project management textbooks.)
Figure 7.4. Example of a simple Gantt chart
It is important to clearly communicate progress both within the project steering committee and to individuals outside the committee. Senior management should be kept apprised of project progress, budget needs, and committee activities. Regular updates should be provided to senior management as well as other user groups involved in the process. Communication can be both formal and informal—everything from periodic update reports at executive meetings to facility newsletter briefings to informal discussions at lunch.
Managing the system acquisition process successfully requires strong and effective leadership, planning, organizational, and communication skills. Things can and do go wrong. Upholding a high level of objectivity and fairness throughout the acquisition process is important to all parties involved. Failing to do so can hamper the overall success of the project. Following is a list of some common pitfalls in the system acquisition process, along with strategies for avoiding them.
These are just a few of the many issues that can arise during the system acquisition process that the health care executive should be aware of. Failing to appropriately address these issues can interfere with the organization’s ability to successfully select and implement a system that will be adopted and widely used.
Acquiring or selecting a new clinical or administrative information system is a major undertaking for a health care organization. It is important that the process be managed effectively. Although the time and resources needed to select a new system will vary depending on the size, complexity, and needs of the organization, certain fundamental issues should be addressed in any system acquisition project.
This chapter discussed the various activities that occur in the system acquisition process. These activities were presented in the context of a multiphysician group practice that wishes to replace its current paper record with an EHR system by acquiring a system from a reputable vendor. Key activities in the system selection process are (1) establishing a project steering committee and appointing a strong project manager to lead the effort, (2) defining project objectives, (3) screening the vendor marketplace, (4) determining system goals, (5) establishing system requirements, (6) developing and administering a request for proposal or request for information, (7) evaluating vendor proposals, and (8) conducting a cost-benefit analysis on the various options. Other options such as contracting with an application service provider (ASP) or a system developer were also discussed. Finally, this chapter presented some of the issues that can arise during the system selection process and outlined the importance of documenting and communicating project activities and progress.
American Health Information Management Association. (2007). The RFP process for EHR systems (updated). Retrieved February 2013 from library.ahima.org/xpedio/groups/public/documents/ahima/bok1_047961.hcsp?dDocName=bok1_047961.
Brooke, J. (1996). SUS: A “quick and dirty” usability scale. In P. W. Jordan, B. Thomas, I. L. McClelland, & B. A. Weerdmeester (Eds.), Usability evaluation in industry (pp. 189–194). London: Taylor & Francis.
Corrao, N. J., Robinson, A. G., Swiernik, M. A., & Naeim, A. (2010). Importance of testing for usability when selecting and implementing an electronic health or medical record system. Journal of Oncology Practice, 6(3), 120–124.
DeLuca, J., & Enmark, R. (2002). The CEO’s guide to health care information systems (2nd ed.). San Francisco: Jossey-Bass.
Eisenstein, E. L., Jurwishin, D., Kushniruk, A.W., & Nahm, M. (2011). Defining a framework for health information technology evaluation. Studies in Health Technology and Informatics, 164, 94–99.
Fortin, J., & MacDonald, K. (2006). Physician practices: Are application service providers right for you? Oakland: California HealthCare Foundation.
Institute of Medicine. (2011). Health IT and patient privacy: Building safer systems for better care. Washington, DC: National Academies Press.
Jones, S. S., Koppel, R., Ridgley, M. S., Palen, T., Wu, S., & Harrison, M. I. (2011, August). Guide to reducing unintended consequences of electronic health records. Rockville, MD: Agency for Healthcare Research and Quality.
Lewis, J. R., & Sauro, J. (2009). The factor structure of the system usability scale. In Proceedings of the Human Computer Interaction International Conference (HCII 2009), San Diego, CA.
Oz, E. (2012). Management information systems: Instructor edition (6th ed.). Boston: Course Technology.
Wager, K. A., & Lee, F. W. (2006). Introduction to healthcare information systems. In M. Johns (Ed.), Health information management technology: An applied approach (2nd ed.). Chicago: American Health Information Management Association.