Open access peer-reviewed chapter

Owning Attention: Applying Human Factors Principles to Support Clinical Decision Support

Written By

Robin Littlejohn, Ronald Romero Barrientos, Christian Boxley and Kristen Miller

Submitted: 05 April 2019 Reviewed: 27 March 2020 Published: 26 April 2020

DOI: 10.5772/intechopen.92291

From the Edited Volume

Recent Advances in Digital System Diagnosis and Management of Healthcare

Edited by Kamran Sartipi and Thierry Edoh

Chapter metrics overview

784 Chapter Downloads

View Full Metrics

Abstract

In the best examples, clinical decision support (CDS) systems guide clinician decision-making and actions, prevent errors, improve quality, reduce costs, save time, and promote the use of evidence-based recommendations. However, the potential solution that CDS represents are limited by problems associated with improper design, implementation, and local customization. Despite an emphasis on electronic health record usability, little progress has been made to protect end-users from inadequately designed workflows and unnecessary interruptions. Intelligent and personalized design creates an opportunity to tailor CDS not just at the patient level but specific to the disease condition, provider experience, and available resources at the healthcare system level. This chapter leverages the Five Rights of CDS framework to demonstrate the application of human factors engineering principles and emerging trends to optimize data analytics, usability, workflow, and design.

Keywords

  • clinical decision support
  • CDS Five Rights
  • electronic health record
  • human factors engineering
  • user-centered design

1. Current state of clinical decision support

Clinical decision support (CDS), leveraging features within the electronic health record (EHR), is increasingly recognized as a valuable tool for providing cognitive support for diagnosis, severity assessment, clinical management, and disposition. CDS is defined as “providing clinicians with clinical knowledge and patient-related information, intelligently filtered, and presented at appropriate times to enhance patient care” [1, 2]. In the best examples, CDS systems guide clinician decision-making and actions [3], prevent errors [4, 5], improve quality [6, 7], reduce costs [8], save time [9], and promote the use of evidence-based recommendations [10]. CDS has the potential to enable clinicians to better address rising information needs, providing the opportunity to pick up on subtle early indications of risk or vulnerability while sorting through an avalanche of data. The availability of evidence-based guidelines for clinical care and for CDS implementation encourages providers to deliver the best, evidence-based care available.

The potential solution that CDS represents is limited by problems associated with improper design, implementation, and local customization. The interaction of poorly designed technologies, organizational constraints, and lowered functional capability has the potential to multiply latent risks in healthcare technologies. Computerized systems that are designed to help clinicians make decisions fail two-thirds of the time as a result of factors such as providing incorrect information or providing information at a point that is incompatible with the workflow [11]. There are surprisingly low acceptance rates for some forms of CDS; approximately 91% of real-time CDS is overridden or ignored by clinicians due to time constraints, perceived misleading alerts, or their patients did not meet certain criteria (such as age or condition) [12]. High rates of alert overrides have been widely acknowledged as a deterrent to acceptance and appropriate use of CDS [13]. Alert overload is detrimental to clinician performance, not only because it can lead to errors by overriding true positive alerts, but also because the false alerts consume clinicians’ time and mental resources. The overabundance of pop-ups, notifications, and check-boxes is highly distracting and produces sensory overload and a perception of extra work without value which contributes to the development of negative perceptions of health information technology. These negative perceptions contribute to low job satisfaction, early retirement, and high turnover [14]. As a result, research indicates that the use of automated, and real-time alerts are only modestly effective in increasing the performance of key tasks [15].

Backed by sophisticated analytics and algorithms to advance clinical decision-making, coupled with increasing pressure to increase throughput and reduce costs, the EHR is often thought to be the solution to the deadly problems of adverse events and inappropriate prescribing. However, the EHR often provides alerts that are perceived by the physicians as unnecessary and clinically insignificant, contributing to alert fatigue and provider burnout [16]. Despite an emphasis on EHR usability, little progress has been made to protect end-users from inadequately designed workflows and unnecessary interruptions [17]. Clinicians’ lack of motivation to use CDS appears to be related to the perceived value of the function combined with the lack of integration into workflow [18]. By identifying factors that predict clinically insignificant alerts and inappropriate responses, informatics personnel can improve alert logic to account for factors such as workflow and patient complexity, increasing specificity of alerts. As a result of the improved specificity, clinicians may experience less alert fatigue, override fewer alerts, and provide better care for patients with conditions that warrant serious alerts. The ultimate goal is to integrate clinical research with human factors engineering to develop optimized CDS systems to satisfy the information needs of clinicians as they formulate, debate, and discuss next steps in treatment or diagnostics for patients. CDS interventions improve care processes and outcomes when they achieve the CDS Five Rights—i.e., deliver the right information to the right people using the right formats via the right channels at the right times in the workflow [19]. This chapter leverages the Five Rights of CDS framework to describe good and bad examples of CDS design, development, and implementation; demonstrate the application of human factors engineering principles to CDS; and describe emerging trends to optimize data analytics, usability, workflow, and design.

Advertisement

2. Application of human factors engineering

Human factors engineering (HFE) is an established scientific discipline used in many high-reliability organizations. HFE takes a system approach to identify crucial components of the man-machine interface and human interactions such as communication, teamwork, and culture. By acknowledging human limitations and system vulnerabilities, HFE minimizes and mitigates human frailties to optimize system performance [20]. Given the complexity of healthcare systems and processes, current HFE healthcare research emphasizes the need for increasing partnerships between human factors engineers and clinical medicine to enhance the standard of care through in-depth evaluation and thoughtful system redesign. Human factors principles, standards, and guidelines provide considerations for the design and development of CDS.

Human factors principles suggest that the format and presentation of the CDS may not be readily applied in the busy acute clinical setting and fail to provide confidence to clinical staff. Effective presentation of an alert, including how and what is displayed, may offer better cognitive support during busy patient encounters and may help providers extract information quickly. Following good human factors principles, alerts should signal to an important matter, inform, and guide the provider [21]. Traditionally, alerts are system components that serve to direct a user’s attention to information related to a value that has exceeded a parameter threshold [22]. Newer alerts, however, have advanced to the point of becoming a “type of automation that supplements the human powers of observation and decision” [23]. Alerts amplify the capacity of clinicians to continuously monitor changes in patient status and thereby support timely intervention. Alerts should be prioritized according to the severity of consequence that could be prevented by taking corrective action (severity) and according to the time available for successful corrective action to be performed (urgency). Substantial human factors analysis remains to be done to realize the potential benefits of CDS.

Advertisement

3. Five Rights of CDS

A useful framework for achieving success in CDS design, development, and implementation is the “CDS Five Rights” approach [24]. The CDS Five Rights model states that we can achieve CDS-supported improvements in desired healthcare outcomes if we communicate: [1] the right information: evidence-based, suitable to guide action, pertinent to the circumstance; [2] to the right person: considering all members of the care team, including clinicians, patients, and their caretakers; [3] in the right CDS intervention format: such as an alert, order set, or reference information to answer a clinical question; [4] through the right channel: for example, a clinical information system such as the EHR, a personal health record (PHR), or a more general channel such as the Internet or a mobile device; [5] at the right time in workflow: for example, at time of decision/action/need. CDS has not reached its full potential in driving care transformation, in part because opportunities to optimize each of the five rights has not been fully explored and cultivated [25].

3.1 Right information

Right information is defined as providing the right information to end users (e.g., clinicians, patients), presenting evidence-based data that is shaped by national clinical guidelines, performance measures, and predictive analytics. The evidence-based data should be relevant to the issue at hand and actionable, meaning the information supports driving clinical actions [19, 24]. The rapid acceleration of technology and the convergence of predictive analytics and human factors address Centers for Medicare and Medicaid Services (CMS) Stage 3: Meaningful Use [26]. The process of integrating real-time analytics into clinical workflow represents a shift towards more agile and collaborative infrastructure building, expected to be a key feature of future health information technology strategies. As interoperability and big data analytics capabilities become increasingly central to crafting the healthcare information systems of the future, the need to address issues that ease the flow of health information and communication become even more important.

Without tools that select, aggregate, and visualize relevant information among the vast display of information competing for visual processing, clinicians must rely on cues by “hunting and gathering” in the EHR. Alerts that embody “right information” should provide just enough data that drives end user action, but not so much data so as to cause alert fatigue [27]. Providing too much information to the end user can spur cognitive overload, with the CDS being ignored or overridden.

In a random sample survey of 300 ambulatory care clinicians using an electronic prescribing system, attitudes towards a drug-drug interaction (DDI) alerting system were measured. In relation to the first of the five CDS rights, relevant and right information, 58% of survey respondents noted dissatisfaction with alerts being triggered by discontinued medications [28]. This dissatisfaction towards alerts that lack usefulness in the clinical environment has been noted by physicians as a potential reason for low rates of alert acceptance [29].

Recent literature has attempted to identify usability design principles relevant to CDS alerting, specifically in the context of medication alerting. Principles that directly relate to right information (first of the Five Rights) include: display relevant data within medication alerts that support the decision-making process, make actionable suggestions based on evidence but do not actively enforce those actions, and provide evidence to support the alerting system (e.g., clinical evidence, patient context, imaging) [30]. The majority of CDS systems assume the diagnostic process is completed accurately and do not provide features beyond general alerts, reminders, summary dashboards, and automated information retrieval. These solutions are not, in fact, decision support. DDI alerts are frequently used CDS alerts that are created to guide appropriate medication management in patients. DDI alerts are highly disregarded by physicians with 49–96% of safety alerts overridden [31] by the physician, which could possibly be due to alert fatigue. Alert fatigue occurs due to a variety of reasons, with patient specificity playing a role. DDI alerts that do not incorporate patient specific context hold varying levels of significance. “An interaction of little relevance to one patient may be of great relevance to another.” In an ideal alert, specific patient context would be used to tailor the specific presentation of these DDI alerts based on age, comorbidities, or medication history [32].

Future innovative approaches will enhance CDS to quantify uncertainty in diagnostic problem solving and present the clinician with additional information regarding probability and likelihood of a diagnosis in the context of diagnoses with similar presentations. Diagnostic suggestions and guidelines are integrated as CDS rules that are extracted, rendered, and then delivered through CDS systems to provide clinicians with just-in-time information, for single disease states, assuming the diagnostic process is completed accurately. When the diagnosis is not immediately obvious, clinicians can use differential diagnosis support tools as an aid to rapidly identify diagnostic possibilities, but this method is highly subject to provider bias and requires manual input. More powerful and accurate analytic layers embedded into EHRs might mitigate both the cause and likelihood of errors (e.g., misdiagnosis) and could allow for more rapid, accurate diagnosis. Analytics-driven CDS can highlight areas prone to poor clinical decision making, increase our knowledge about conditions that are vulnerable to being missed and help prioritize diagnostic errors. The ultimate goal is to integrate clinical research, design, and systems engineering to develop optimized CDS systems – satisfying the information needs of clinicians as they formulate, debate, and discuss next steps in the diagnosis and clinical management of patients.

3.2 Right person

To the right person (second of the Five Rights) involves providing CDS interventions to the appropriate parties that have the capabilities to take appropriate action. Possibilities of “the right person” include: care team members, clinicians & care providers, patients & patient caretakers [16, 17].

Usability design principles related to the right person include displaying alerts to primary clinicians, as well as clinicians who do not have primary responsibility to serve as a second check. This is done by indicating to all professionals involved in the patient’s care that there is information available, as well as informing the relevant care team as to how previous alerts were handled, if documented. Additional strategies include system-level alerts like the Rothman Index “quilt view” for alerting [33]. The Rothman Index is a comprehensive rating of overall patient condition in the hospital setting. The index is used at many medical centers and calculated based on vital signs, laboratory values, and nursing assessments in the EHR. In addition to patient-specific CDS, the “quilt view” assigns a risk color to each patient, providing an overall indicator of a unit’s condition, available for viewing by both clinicians with primary responsibility of the patient and unit leadership (e.g., a unit’s charge nurse).

Providing correct information is not limited to only clinicians but can be extended to the patient. Patient decision support interventions are attempts to use CDS to translate medical knowledge to patients. Patient facing CDS assist patients in gaining a better understanding and comprehension of medical decisions. Patient facing CDS shows great promise in assisting shared decision making between physician and provider, but there are still obstacles to overcome regarding implementation. A recent systematic review of literature related to implementation of patient decision support interventions have shown that there are administrative and clinical challenges in implementation of these patients facing CDS [34].

There is a significant push toward providing the important and informative information to the appropriate users in the clinical settings in a consistent and usable manner. The Agency for Healthcare Research and Quality (AHRQ) funded demonstration projects that yielded important knowledge about translating narrative guidelines into formats that can be used by EHRs, and about implementing CDS in clinical settings [35]. AHRQ also acknowledge a range of important research questions still to be answered in the areas of guideline translation, local CDS implementation, clinician and patient factors that affect success, and policy and sustainability issues. Rather than replicate the technical advances that have been made in the field, future research will focus on translating a CDS tool into a framework oriented towards streamlining creation, implementation, and dissemination. Most of the “work” performed by everyday clinicians for patients is highly individualized. Thus, a deep understanding of the local, highly personal context is required to get CDS “right.” Moreover, getting CDS “wrong” will not be the equivalent of not providing any CDS. Rather, there is a real risk of inefficiency (e.g., interruption and distraction, leading the clinician to forget what they were thinking about before the CDS) and patient harm (e.g., acceptance of CDS that is inappropriate given the specific patient’s clinical situation).

3.3 Right format

Alert and warning complexity are especially prevalent in health information technology. Despite this issue, there is little consensus on how alerts should be generated and displayed to the user [36] as well as what level of interaction is appropriate. CDS may be implemented in various formats (e.g. alerts, order sets, protocols, patient monitoring systems, info buttons). Consequently, it becomes important for implementers to identify the issues and problems they are trying to solve and choose the best format to resolve the problem at hand (third of the Five Rights). Furthermore, when developing a CDS program, implementers should create an inventory of current systems to determine which CDS tools are available, which tools need to be developed in-house, and which tools need to be purchased through a vendor. Specifically, there is a lack of knowledge regarding the most effective ways to differentiate alerts, highlighting important pieces of information without adding noise, to create a universal standard [37]. While underlying models and algorithms of CDS have been intensively studied, there remains a lack of evidence-based guidelines in terms of functional and design requirements of the system.

The purpose of an alert is to prompt an operator action. Poor alert system design has been a contributing cause of adverse events in numerous healthcare systems worldwide. The appeal of access to a large amount of clinical data must be balanced against the real possibility of information overload. Research demonstrates that medical displays are often incompatible with practitioners’ workflow and unnecessarily fragment patient information [38]. Information is often spread across multiple tabs and locations that require piecemeal information search and acquisition. This may confound practitioners’ ability to detect evolving changes, make it more difficult to attain a holistic view of a patient’s health state, lead to care inefficiencies, and frustrate clinicians. Recognizing limits on human working memory, clinicians need external automated information systems to support early detection of patient deterioration and improve timeliness of therapeutic response. The design of alerts must improve the process of information display, reducing cognitive load on the working memory of the provider and improving the usual process that is often characterized by fragmented, non-directed information gathering.

In the absence of evidence-based guidelines specific to EHR alerting, effective alert design can be informed by several guidelines for design, implementation, and reengineering that help providers take the correct action at the correct time in response to recognition of the patient’s condition [39]. In a narrative review to inform EHR alert optimization and clinical practice workflow, 42 unique recommendations were included and classified as interface, information, and interaction features [39]. The recommendations identified are described to help optimize design, organization, management, presentation, and utilization of information through presentation, content, and function. Alarm systems should be designed for and driven by human factors rather than technical capabilities. Easterby suggests seven psychological processes to be considered in display design that determine the limits of display formats: [1] detection – determining the presence of an alarm; [2] discrimination – defining the differences between one alarm and another; [3] identification – attributing a name of meaning to an alarm; [4] classification – group the alarms with a similar purpose of function; [5] recognition – knowing what an alarm purports to mean; [6] scaling – assigning values to alarms; [7] ordering and sequencing – determining the relative order and priority of alarms [40].

3.4 Right channel

Not only must a CDS provide information to the correct information to the appropriate audience in a usable format, the CDS must provide the information via the most effective and efficient channel (fourth of the Five Rights). CDS interventions can be delivered through an EHR, PHR, computerized physician order entry, an app running on a smartphone, and—if necessary—in paper form via flow-sheets, forms, and labels. Typically, organizations deploying CDS will focus very heavily on interruptive alerts—especially to physicians via computerized physician order entry (CPOE). Although alerts can be a powerful CDS intervention, they tend to be used excessively and inappropriately, resulting in commonly reported CDS problems such as alert overrides, physician frustration, and backlash. If the physician is the right person, then the EHR may be the best platform for delivering the alert. However, if a significant other is the right person, then the right platform may be a text messaging app running on a smartphone.

Emerging trends include the use of mobile technology and patient portals as CDS channels. Healthcare systems are gradually moving toward new models of care based on integrated care processes shared by various care givers and on an empowered role of the patient. Good communication between patients, their providers and caregivers improve patient satisfaction and are central to optimal outcomes. The explosion of mobile technologies and healthcare applications represents a growing opportunity to optimize care delivery. Availability of medical information through internet-enabled smartphones and tablets has increased significantly. These applications provide medical providers with recommendations for treatment, disposition, and prescriptions conforming to the most up-to-date evidence-based guidelines; allow instant access to journals and information sources at the click of a button; and deliver a plethora of CDS tools. Patients are also using the technology to communicate with their providers, research medical conditions, and become more active participants in their care. Mobile applications illustrating complex medical conditions and processes and online resources are being recommended by physicians to aid the patient in this role.

3.5 Right workflow

The clinical space is polluted with alerts that are unheeded. Despite the theoretical promise of CDS systems, their development and successful implementation is poorly managed [41]. Right workflow (fifth of Five Rights) is defined as making sure information is presented at the right time and is available when needed. For example, passive alerts can appear in a prominent place in the EHR – a decision based on the results of a workflow analysis – and can be processed once the physician completes the physical examination. An alternative method would be when the physician closes the patient record they are given a prompt informing them there are outstanding patient alerts that need to be processed. The application of human factors in determining the right workflow includes but is not limited to ethnographic research include workflow analysis and usability testing.

Workflow analysis is a process in which researchers examine the progression of workflows to improve efficiency. Ethnographic research is a qualitative method where researchers observe and/or interact with system users in their real-life environment. Observation is a systematic data collection approach by which information is gathered by watching behavior, events, and people in natural settings and naturally occurring situations. User observation is unique in that it combines the researcher’s participation in the lives of the people and processes under study while also maintaining a professional distance [42]. According to Angrosino, “Observation is the act of perceiving the activities and interrelationships of people in the field setting” [43]. The demand for usability testing is becoming increasingly important as healthcare moves toward a commitment to the Triple Aim: improving the experience of care, improving the health of populations, and reducing per capita costs of healthcare. Usability testing is a critical step in informing and helping define the standard of care for healthcare system, promoting safe, high-quality care for patients. It provides the opportunity to assess user behavior, interaction, and performance data to measure how the design of medical devices, equipment, practices, and protocols affect performance, quality, workflow (cognitive and clinical), and patient safety [44, 45]. The goal is to provide evidence to support the selection and implementation of safe and user-friendly CDS, inform decision-making, and develop better solutions that update, unify, and generally improve the usability of healthcare providers’ tools and systems related to optimal diagnosis and clinical management.

Advertisement

4. Scratching the surface of better CDS

Recent literature supports that using the Five Rights of CDS framework as a foundation for CDS design, development, and implementation can have positive impacts on CDS acceptance as well as positive outward reaching effects on clinician workflow, improved patient care, and increased patient safety.

To provide a useful, standardized, and evidence-based diagnostic aid, Kharbanda et al. developed a CDS tool to aid in the evaluation and management of treatment care for pediatric patients with suspected appendicitis in the emergency department (ED). The CDS took a three-component approach, combining: a standardized abdominal pain medication order set; a web-based stratification tool used to classify the pediatric patient as low, medium, or low-risk for appendicitis; and a “time of ordering” alert with steps for treatment and imaging guidance (e.g., medication and imaging options) for the identified level of risk [46]. The implementation of an evidenced based CDS reduced the number of costly computed tomography (CT) imaging, potentially reducing the number of unnecessary radiation exposure to developing children.

Using a Bluetooth enabled blood glucose (BG) meter in conjunction with a cloud-based clinical decision support system (CDSS), clinicians were able to increase efficiency and efficacy of glucose monitoring in diabetic patients [47]. The appropriate technology enabled patients to increase self-monitoring. Clinicians were able to more closely monitor patient’s BG readings and suggest insulin dose and titration changes between appointments, as applicable, using the patient’s BG meter, text message, or phone call. Use of the CDSS aided the patient’s care team to increase efficiency in their workflow and provide improved patient care regarding getting patients within target glucose ranges.

In addressing the deficiencies of appropriate medication ordered for patients with impaired renal function and the lack of re-assessment of medication appropriateness as patients’ symptoms change, Awdishu et al. developed a dynamic CDS tool within an EHR that provides renal medication dosing suggestions and alternative therapies suggestions at the initial time of medication prescription (“prospective alert”) and temporal alerts during continuous monitoring of patients’ renal function (“look-back alerts”) [48]. All alerts only fired during the order entry workflow (i.e., at the point of placing and/or updating a medication order). Study results indicate the alerts had a significant impact on the selection of appropriate drug prescription during medication initiation, in addition to significant improvements for appropriate medication adjustments.

Exploring additional channels for CDS, Burgess et al. evaluated the impact of an online care processing models (CPM), on the quality of care for patients with lower extremity cellulitis (LEC). When the CPM was utilized, there was an increasing trend in appropriate drug prescription during medication initiation and at patient discharge [49].

Despite continued growth and successful implementation of CDS tools, CDS has not reached its full potential in driving health care transformation [25]. Opportunities to optimize each of the five rights continue to be highlighted by challenges and barriers such as gaining full acceptance from users from various disciplines, cultures, and use settings; continually maturing technology standards that restrict cohesive integration; and the growing resource requirements needed to keep customized solutions up to date [50].

Advertisement

5. Conclusions

Human factors approach underpins patient safety and quality improvement science, offering an integrated, evidence-based, coherent approach to improving the science behind health care delivery. Improvements in display management have commenced, but there is great need for further progress. As demands on healthcare providers increase (the result of increasing availability and complexity of medical devices and delivery processes, higher patient illness acuity, higher costs for process interruptions), the potential for problems are increasing. Safety-critical interactions with the EHR are especially common, challenging, and important. In safety critical environments (such as hospitals), the importance of well-designed, usable interfaces is increased precisely because of the potential for catastrophic outcomes. Time pressure, competing demands, and ambiguous alert design reduce a user’s opportunity to detect signals in the face of noise and may lead to inadvertent confirmation bias. The importance of and need for appropriate user interface design is increasingly evident in such environments.

The efficiency of alert design depends on several guidelines for design, implementation, and reengineering that help providers to take the correct action at the correct time in response to recognition of the patient’s condition. Hollifield proposed the following six guidelines for alert development: [1] alarms are properly chosen and implemented; [2] alerts are relevant, clear, and easy to understand; [3] operators can rapidly assess the relative importance of alerts; [4] operators can process alert information during high frequency events; [5] priority determination; and [6] alert management enhances the operator’s ability to make a judgment based on experience and skill [51]. Stanton and Stammer place importance on alert prioritization and organization, which impact early detection of critical alerts [52]. Information must be presented so it is compatible with human capabilities and limitations, so that the system remains usable for the provider in all situations [21]. We considered aspects of display design in relation to taxonomy of provider psychological process that illustrate the different nature of the two types of enhanced visual display models developed for this research.

CDS can be utilized across a variety of conditions and circumstances to promote optimal care. CDS has ultimately improved adherence to recommended care standards and may result in lasting improvements in the clinical setting [53, 54]. However, the accuracy and acceptance of CDS can be limited by numerous factors, including poor usability and too many false positive alerts. There is growing evidence that health information technology interventions ultimately improve patient outcomes through early diagnosis and recommendations of evidence-based protocols [55]. The world of cognitive support is promising due the innovation and growth in this area of study.

Advertisement

Conflict of interest

The authors declare no conflict of interest.

References

  1. 1. Larosa JA, Ahmad N, Feinberg M, Shah M, Dibrienza R, Studer S. The use of an early alert system to improve compliance with sepsis bundles and to assess impact on mortality. Critical Care Research and Practice. 2012;2012:980369
  2. 2. Bailey TC, Chen Y, Mao Y, Lu C, Hackmann G, Micek ST, et al. A trial of a real-time alert for clinical deterioration in patients hospitalized on general medical wards. Journal of Hospital Medicine. 2013;8(5):236-242
  3. 3. Garg AX, Adhikari NKJ, McDonald H, Rosas-Arellano MP, Devereaux PJ, Beyene J, et al. Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: A systematic review. Journal of the American Medical Association. 2005;293:1223-1238
  4. 4. Ammenwerth E, Schnell-Inderst P, Machan C, Siebert U. The effect of electronic prescribing on medication errors and adverse drug events: A systematic review. Journal of the American Medical Informatics Association. 2008;15(5):585-600
  5. 5. Wolfstadt JI, Gurwitz JH, Field TS, Lee M, Kalkar S, Wu W, et al. The effect of computerized physician order entry with clinical decision support on the rates of adverse drug events: A systematic review. Journal of General Internal Medicine. 2008;23(4):451-458
  6. 6. Kawamoto K, Houlihan CA, Balas EA, Lobach DF. Information in practice systematic review of trials to identify features critical to success. BMJ. 31 March 2005;330(7494):765
  7. 7. Field TS, Rochon P, Lee M, Gavendo L, Baril JL, Gurwitz JH. Computerized clinical decision support during medication ordering for long-term care residents with renal insufficiency. Journal of the American Medical Informatics Association. 2009;16(4):480-485
  8. 8. Fischer MA, Vogeli C, Stedman M, Ferris T, Brookhart MA, Weissman JS. Effect of electronic prescribing with formulary decision support on medication use and cost. Archives of Internal Medicine. 2008;168(22):2433-2439
  9. 9. Murphy EV. Clinical decision support: Effectiveness in improving quality processes and clinical outcomes and factors that may influence success. The Yale Journal of Biology and Medicine. 2014;87:187-197
  10. 10. Eslami S, Abu-Hanna A, de Keizer NF. Evaluation of outpatient computerized physician medication order entry systems: A systematic review. Journal of the American Medical Informatics Association. 2007;14(4):400-406
  11. 11. Hussey PS, Timbie JW, Burgette LF, Wenger NS, Nyweide DJ, Kahn KL. Appropriateness of advanced diagnostic imaging ordering before and after implementation of clinical decision support systems. JAMA. 2015;313(21):2181-2182
  12. 12. Acquadro C, Berzon R, Dubois D, Leidy NK, Marquis P, Revicki D, et al. Incorporating the patient’s perspective into drug development and communication: An ad hoc task force report of the patient-reported outcomes (PRO) Harmonization Group meeting at the food and drug administration, February 16, 2001. Value Health. September-October 2003;6(5):522-531
  13. 13. Strom BL, Schinnar R, Aberra F, Bilker W, Hennessy S, Leonard CE, et al. Unintended effects of a computerized physician order entry nearly hard-stop alert to prevent a drug interaction: A randomized controlled trial. Archives of Internal Medicine. 2010;170(17):1578-1583
  14. 14. Hysong SJ, Spitzmuller C, Espadas D, Sittig DF, Singh H. Electronic alerts and clinician turnover: The influence of user acceptance. The American Journal of Managed Care. 2014;20(11):SP520-SP530
  15. 15. Sittig DF, Krall MA, Dykstra RH, Russell A, Chin HL. A survey of factors affecting clinician acceptance of clinical decision support. BMC Medical Informatics and Decision Making. 2006;1:6
  16. 16. Halbesleben JRB, Rathert C. Linking physician burnout and patient outcomes: Exploring the dyadic relationship between physicians and patients. Health Care Management Review. 2008;33(1):29-39
  17. 17. Ratwani RM, Benda NC, Zachary Hettinger A, Fairbanks RJ. Electronic health record vendor adherence to usability certification requirements and testing standards. JAMA : The Journal of the American Medical Association. 2015;314:1070-1071
  18. 18. Berner ES. Clinical decision support systems: State of the art. AHRQ Publication 90069. 2009. pp. 1-26
  19. 19. Osheroff J, Teich JM, Levick D, Saldana L, Velasco F, Sittig DF, et al. Improving Outcomes with Clinical Decision Support: An Implementer’s Guide. 2nd ed. Chicago: CRC Press; 2012. p. 323
  20. 20. Carayon P, Wood KE. Patient safety - the role of human factors and systems engineering. Studies in Health Technology and Informatics. 2010;153:23-46
  21. 21. Noyes J. Engineering Equipment and Materials Users’ Association. Alarm Systems: A Guide to Design, Management and Procurement. Computing and Control Engineering. 2000;11(2):52
  22. 22. The Computer-Based Patient Record [Internet]. Washington, D.C.: National Academies Press. 1997. Available from: http://www.nap.edu/catalog/5306
  23. 23. Pritchett AR, Vándor B, Edwards K. Testing and implementing cockpit alerting systems. Reliability Engineering and System Safety. 2002;75(2):193-206
  24. 24. Campbell JR. The five rights of clinical decision support: CDS tools helpful for meeting meaningful use. Journal of AHIMA. 2013;84(10):42-47
  25. 25. Sittig DF, Wright A, Osheroff JA, Middleton B, Teich JM, Ash JS, et al. Grand challenges in clinical decision support. Journal of Biomedical Informatics. 2008;41(2):387-392
  26. 26. McNeill D, Davenport TH. Analytics in Healthcare and the Life Sciences: Strategies, Implementation Methods, and Best Practices. Upper Saddle River: Pearson; 2014
  27. 27. Woods DD. The alarm problem and directed attention in dynamic fault management. Ergonomics. 1995;38(11):2371-2393
  28. 28. Weingart SN, Simchowitz B, Shiman L, Brouillard D, Cyrulik A, Davis RB, et al. Clinicians’ assessments of electronic medication safety alerts in ambulatory care. Archives of Internal Medicine. 2009;169(17):1627-1632
  29. 29. Isaac T, Weissman JS, Davis RB, Massagli M, Cyrulik A, Sands DZ, et al. Overrides of medication alerts in ambulatory care. Archives of Internal Medicine. 2009;169(3):305-311
  30. 30. Marcilly R, Ammenwerth E, Roehrer E, Niès J, Beuscart-Zéphir MC. Evidence-based usability design principles for medication alerting systems. BMC Medical Informatics and Decision Making. 2018;18(1):69
  31. 31. Van Der Sijs H, Aarts J, Vulto A, Berg M. Overriding of drug safety alerts in computerized physician order entry. Journal of the American Medical Informatics Association. 2006;13(2):138-147
  32. 32. Duke JD, Bolchini D. A successful model and visual design for creating context-aware drug-drug interaction alerts. AMIA Annual Symposium Proceedings. 2011;2011:339-348
  33. 33. Rothman SI, Rothman MJ, Solinger AB. Placing clinical variables on a common linear scale of empirically based risk as a step towards construction of a general patient acuity score from the electronic health record: A modelling study. BMJ Open. 2013;3(5):e:002367
  34. 34. Elwyn G, Scholl I, Tietbohl C, Mann M, Edwards AG, Clay C, et al. “Many miles to go.”: A systematic review of the implementation of patient decision support interventions into routine clinical practice. BMC Medical Informatics and Decision Making. 2013;13(S2):S14
  35. 35. Mardon R, Mercincavage L, Johnson M Jr, Finley S, Pan E, Arora D. Findings and Lessons From AHRQ’s Clinical Decision Support Demonstration Projects. (Prepared by Westat under Contract No. HHSA 290-2009-00023I). Rockville, MD; 2014. Report No.: AHRQ Publication No. 14-0047-EF
  36. 36. Phansalkar S, Edworthy J, Hellier E, Seger DL, Schedlbauer A, Avery AJ, et al. A review of human factors principles for the design and implementation of medication safety alerts in clinical information systems. Journal of the American Medical Informatics Association. 2019;17(5):493-501
  37. 37. Miller RA, Waitman LR, Chen S, Rosenbloom ST. The anatomy of decision support during inpatient care provider order entry (CPOE): Empirical observations from a decade of CPOE experience at Vanderbilt. Journal of Biomedical Informatics. 2005;38:469-485
  38. 38. Kong N, Heer J, Agrawala M. Perceptual guidelines for creating rectangular treemaps. IEEE Transactions on Visualization and Computer Graphics. 2010;16(6):990-998
  39. 39. Miller K, Mosby D, Capan M, Kowalski R, Ratwani R, Noaiseh Y, et al. Interface, information, interaction: A narrative review of design and functional requirements for clinical decision support. Journal of the American Medical Informatics Association. 2018;25(5):585-592
  40. 40. Easterby R. Tasks, processes and display design. In: Easterby R, Zwaga H, editors. Information Design. Wiley Chichester, UK: Chichester; 1984. pp. 19-36
  41. 41. Gagliardi AR, Alhabib S, Members of Guidelines International Network Implementation Working Group. Trends in guideline implementation: A scoping systematic review. Implementation Science. 2015;10:54
  42. 42. Fetterman DM. Ethnography: Step by Step. SAGE; 1998. p. 165
  43. 43. Angrosino MV. Doing Ethnographic and Observational Research. Los Angeles, [Calif.]; London: SAGE; 2007. Available from: https://search.ebscohost.com/login.aspx?direct=true&db=cat00012a&AN=bourne.910143&site=eds-live&scope=site
  44. 44. Mohammed S, Fiaidhi J. Ubiquitous Health and Medical Informatics: The Ubiquity 2.0 Trend and Beyond. Hershey, Medical Information Science Reference; 2010
  45. 45. Bien N, Rajpurkar P, Ball RL, Irvin J, Park A, Jones E, et al. Deep-learning-assisted diagnosis for knee magnetic resonance imaging: Development and retrospective validation of MRNet. PLoS Medicine. 2018;15(11):e1002699
  46. 46. Kharbanda AB, Madhok M, Krause E, Vazquez-Benitez G, Kharbanda EO, Mize W, et al. Implementation of electronic clinical decision support for pediatric appendicitis. Pediatrics. 2016;137(5)
  47. 47. Bode B, Clarke JG, Johnson J. Use of Decision Support Software to Titrate Multiple Daily Injections Yielded Sustained A1c Reductions After 1 Year [Internet]. 2019. Available from: https://www.glytecsystems.com/Evidence/use-of-decision-support-software-to-titrate-multiple-daily-injections-yielded-sustained-a1c-reductions-after-1-year-jdst.html
  48. 48. Awdishu L, Coates CR, Lyddane A, Tran K, Daniels CE, Lee J, et al. The impact of real-time alerting on appropriate prescribing in kidney disease: A cluster randomized controlled trial. Journal of the American Medical Informatics Association. 2016;23(3):609-616
  49. 49. Burgess MJ, Enzler MJ, Kashiwagi DT, Selby AJ, Sohail MR, Daniels PR, et al. Clinical study of an online tool for standardizing hospital care. Journal for Healthcare Quality. 2016;38(6):359-369
  50. 50. Das M, Eichner J. Challenges and Barriers to Clinical Decision Support (CDS) Design and Implementation Experienced in the Agency for Healthcare Research and Quality CDS Demonstrations (Prepared for the AHRQ National Resource Center for Health Information Technology under C. Rockville), MD; 2010. Report No.: AHRQ Publication No. 10-0064-EF
  51. 51. Hollifield B, Habibi E. Alarm Management: A Comprehensive Guide, Second Edition [Internet]. International Society of Automation; Second edition. 2010. Available from: http://www.amazon.com/Alarm-Management-Comprehensive-Second-Edition/dp/193600755X
  52. 52. Stanton NA, Stammers RB. Alarm initiated activities: Matching formats to tasks. International Journal of Cognitive Ergonomics. 1998;2(4):331-348
  53. 53. Shiffman RN, Liaw Y, Brandt CA, Corb GJ. Computer-based guideline implementation systems. Journal of the American Medical Informatics Association. 1999;6(2):104-114
  54. 54. Oxman AD, Thomson MA, Davis DA, Haynes RB. No magic bullets: a systematic review of 102 trials of interventions to improve professional practice. CMAJ. 1995;153, 10:1423-1431
  55. 55. Chaudhry B, Wang J, Wu S, Maglione M, Mojica W, Roth E, et al. Systematic review: Impact of health information technology on quality, efficiency, and costs of medical care. Annals of Internal Medicine. 2006;144:742-752

Written By

Robin Littlejohn, Ronald Romero Barrientos, Christian Boxley and Kristen Miller

Submitted: 05 April 2019 Reviewed: 27 March 2020 Published: 26 April 2020