Overview: This project built on an interdisciplinary research network, and:
- examined the value of Computerized Provider Order Entry (CPOE) technology from the perspective of: patient safety, quality of care, and financial outcomes;
- used human factors analysis methods for improving the design and implementation of CPOE, and;
- examined the impact of CPOE on end users.
Aim #1: The project determined the effect of CPOE on safety and quality of care in six ICUs by examining the impact of CPOE on medication errors and Adverse Drug Events (ADEs) and a variety of measures of quality of care, including the relation between length of stay and mortality.
Aim #2: The project determined the impact of CPOE on physicians, nurses, pharmacists, and respiratory therapists (all end users) in ICUs. Specific measures related to end users’ perceptions of and attitudes about CPOE and the CPOE change process will include:
• CPOE Characteristics: user-friendliness, usability, expectations and attitudes toward technology;
• Job Characteristics: job control, workload, challenge, role ambiguity;
• Quality of Working Life (QWL): job satisfaction, stress, organizational commitment and safety/quality of care; and
• CPOE Change Process: employee involvement and participation, feedback, training and learning.
Aim #3: Examined the use of prospective human factors analysis in CPOE implementation. The research team used proactive work flow analysis, risk assessment, and usability evaluations to identify changes in CPOE design and implementation.
Documents:
Final Project Report
Project Summary
Definition of a Medication Order
Pascale Carayon, PhD
Procter & Gamble Bascom Professor in Total Quality
Department of Industrial and Systems Engineering
Director, Center for Quality and Productivity Improvement
University of Wisconsin-Madison
Kenneth Wood, DO
Chief Medical Officer
Geisinger Medical Center, Danville, PA
James Walker, MD, FACP
Formerly Chief Health Information Officer
Geisinger Health System, Danville, PA
Peter Hoonakker, PhD
Research Scientist, Associate Director of Research
Center for Quality and Productivity Improvement
University of Wisconsin-Madison
Ann Schoofs Hundt, PhD
Research Scientist, Associate Director of Education
Center for Quality and Productivity Improvement
University of Wisconsin-Madison
Tosha Wetterneck, MD, MS
Associate Professor of Medicine
Center for Quality and Productivity Improvement
University of Wisconsin-Madison
Roger Brown, PhD
Professor, School of Nursing
Director, Research Design Methodology and Statistics Unit
University of Wisconsin-Madison
Randi Cartmill, MS
Researcher, Center for Quality and Productivity Improvement
University of Wisconsin-Madison
Graduate Student Team Members
Kerry McGuire, PhD
Bonnie Paris, PhD
Kara Schultz, PhD
2017
Carayon, P., Wetterneck, T.B., Cartmill, R., Blosky, M.A., Brown, R.L., Hoonakker, P.L.T., Kim, R., Kukreja, S., Johnson, M., Paris, B.L., Wood, K.E., Walker, J.M. (2017). Medication Safety in Two ICUs of a Community Teaching Hospital after EHR Implementation. Journal of Patient Safety. PMCID: PMC5573668
2015
Carayon, P., Wetterneck, T.B., Alyousef, B., Brown, R.L., Cartmill, R.S., McGuire, K., Hoonakker, P.L.T., Slagle, J., Schultz van Roy, K., Walker, J.M., Weinger, M.B., Xie, A., & Wood, K.E. (2015). Impact of electronic health records technology on the work and workflow of physicians in the intensive care units. International Journal of Medical Informatics (IJMI), 84 (8), 578-594. PMCID: PMC4490834
2013
Carayon, P., Wetterneck, T.B., Cartmill, R.S., Blosky, M.A., Brown, R.L., Kim, R., Kukreja, S., Johson, M.A., Paris, B., Wood, K.E., & Walker, J. (2013). Characterising the complexity of medication safety using a human factors approach: An observational study in two intensive care units. BMJ Quality & Safety, 23 (1), 56-65. PMCID: PMC3938094
Douglas, S., Cartmill, R.S., Brown, R., Hoonakker, P., Slagle, J., Van Roy, K., Walker, J.M., Weinger, M.B., Wetterneck, T., & Carayon, P. (2013). The work of adult and pediatric intensive care unit nurses. Nursing Research, 62 (1), 50-58. PMID: 23222843
Hundt, A.S., Adams, J.A., Schmid, J.A., Musser, L.M., Walker, J., Wetterneck, T.B., Douglas, S.V., Paris, B.L., & Carayon, P. (2013). Conducting an efficient proactive risk assessment prior to CPOE implementation in an intensive care unit.International Journal of Medical Informatics, 82 (1), 25-38. PMCID: PMC3430828
Hoonakker, P., Carayon, P., Brown, R.L., Cartmill, R.S., Wetterneck, T.B., & Walker, J. (2013). Changes in end-user satisfaction with computerized provider order entry over time among nurses and providers in intensive care units.Journal of the American Medical Informatics Association, 20 (2), 252-259. PMCID: PMC3638190
Hoonakker, P., Carayon, P., Walker, J., Brown, R.L., & Cartmill, R.S. (2013). The effects of Computerized Provider Order Entry implementation on communication in Intensive Care Units. International Journal of Medical Informatics, 82 (5), 107-117. PMCID: PMC3624062
Waterson, P., Hoonakker, P. L., & Carayon, P. (2013). Special issue on human factors and the implementation of health information technology (HIT): Comparing approaches across nations. International Journal of Medical Informatics,5(82), 277-280. PMID: 23375708
2012
Cartmill, R.S., Walker, J.M., Blosky, M.A., Brown, R.L., Djurkovic, S., Dunham, D.B., Gardill, D., Haupt, M.T., Parry, D., Wetterneck, T.B., Wood, K.E., & Carayon, P. (2012). Impact of electronic order management on the timeliness of antibiotic administration in critical care patients. International Journal of Medical Informatics, 81 (11), 782-791. PMID: 22947701
2011
Hoonakker, P., Carayon, P., Gurses, A., Brown, R., Khunlertkit, A., McGuire, K., & Walker, J.M. (2011). Measuring workload of ICU nurses with a questionnaire survey: The NASA Task Load Index (TLX). IIE Transactions on Healthcare Systems Engineering, 1 (2), 131-143. PMCID: PMC3388621
Carayon, P., Cartmill, R.S., Blosky, M.A., Brown, R., Hackenberg, M., Hoonakker, P., Hundt, A.S., Norfolk, E., Wetterneck, T.B. & Walker, J. (2011). ICU nurses’acceptance of electronic health records. Journal of the American Medical Informatics Association, 18 (6), 812-819. PMCID: PMC3197984
Hoonakker, P., Cartmill, R.S., Carayon, P., & Walker, J. (2011). Development and psychometric qualities of the SEIPS survey to evaluate CPOE/EHR implementation in ICUs. International Journal of Healthcare Information Systems and Informatics, 6 (1), 51-69. PMCID: PMC3070305
Wetterneck, T., Walker, J., Blosky, M.A., Cartmill, R.S., Hoonakker, P., Johnson, M.A., Norfolk, E., & Carayon, P. (2011). Factors contributing to an increase in duplicate medication order errors after CPOE implementation. Journal of the American Medical Informatics Association, 18 (6), 774-782. PMCID: PMC3198002
Carayon, P., Cartmill, R.S., Blosky, M.A., Brown, R., Hackenberg, M., Hoonakker, P., Hundt, A.S., Norfolk, E., Wetterneck, T.B. & Walker, J. (2011). EHR acceptance by ICU physicians and nurses. Proceedings of the Healthcare Ergonomics Systems and Patient Safety (HEPS) Conference, Oviedo, Spain.
Hoonakker, P., Wetterneck, T., Carayon, P., Cartmill, R.S., & Walker, J. (2011). Drug alerts override from a human factors perspective. Proceedings of the Healthcare Ergonomics Systems and Patient Safety (HEPS) Conference, Oviedo, Spain. pp. 367-371.
Wetterneck, T., Paris, B., Blosky, Walker, J., M.A., Cartmill, R.S., & Carayon, P. (2011). A Method for Evaluating Medication Error Recovery Processes in the ICU. Paper presented at the Healthcare Ergonomics Systems and Patient Safety (HEPS) Conference, Oviedo, Spain.
Wetterneck T.B. (2011). Error recovery in health care. In P. Carayon (Ed.). Handbook of Human Factors and Ergonomics in Health Care and Patient Safety, (2 ed., pp. 449-461). CRC Press: Boca Raton, FL.
2010
Hoonakker, P., Carayon, P. & Walker, J. (2010). Measurement of CPOE end-user satisfaction among ICU physicians and nurses. Applied Clinical Informatics, 1 (3), 268-285. PMCID: PMC3059318
Wetterneck, T.B., Cartmill, R.S., Walker, J.M., Carayon, P. (2010). Duplicate medication orders after CPOE implementation.Journal of Hospital Medicine, 5 (3), 81.
Carayon, P., Faye, H., Hundt, A.S., Karsh, B-T., Wetterneck, T.B. (2010). Patient safety and proactive risk assessment. In Yuehwen (Ed.), Handbook of Healthcare Delivery Systems, 12, (pp. 1-15). Taylor & Francis: London.
Hoonakker, P., Cartmill, R.S., Carayon, P., & Walker, J. (2010). Development of the survey to evaluate CPOE/EHR implementation in ICUs. Technical Report, Center for Quality and Productivity Improvement (CQPI), Madison, WI.
Hoonakker, P., Carayon, P., Wood, K., & Walker, J. (2010). Implementation of healthcare technology and workload of nurses in the intensive care units. Poster presented at the ICOH WOPS Conference, Amsterdam, The Netherlands.
Schneider, K., Nichols, L., Stevens, C., & Hook, J.M. (2010). AHRQ Publication No. 10-0095-3-EF. (Prepared by John Snow, Inc. under contract No. HHSA 290200900018I, TO 3). Success stories from the AHRQ-funded health IT portfolio.
2009
Carayon, P. (2009). EHR-related medication errors and events in ICUs. Paper presented at the Human Factors and Ergonomics Society (HFES) 53rd Annual Meeting, San Antonio, TX.
Djurkovic, S., Cartmill, R.S., Blosky, M.A., Dunham, D., Gardill, D., Haupt, M., Parry, D., Walker, J.M., Wetterneck, T.B., Wood, K.E., & Carayon, P. (2009). CPOE Implementation decreases the time to the administration of the first dose of antibiotic. Paper presented at the 2010 Society for Critical Care Medicine Conference, Miami, FL.
Hoonakker, P., Carayon, P., Cartmill, R.S., Walker, J., & Korunka, C. (2009). Healthcare Information Technology Implementation and Workload, Burnout and Perceived Quality of Care and Patient Safety among Nurses in the ICU.Paper presented at the APA/NIOSH Conference, San Juan, Puerto Rico.
2008
Carayon, P. (2008). Macroergonomics issues in the design, implementation and use of electronic health record in intensive care units. Paper presented at the Human Factors and Ergonomics Society (HFES) 52nd Annual Meeting, New York, NY.
Hoonakker, P., Carayon, P., Douglas, S., Schulz, K., Walker, J., & Wetterneck, T.B. (2008). Communication in the ICU and then relation with quality of care and patient safety from a nurse perspective. Paper presented at the Human Factors and Ergonomics Society (HFES) 54th Conference, San Franscisco, CA.
Hoonakker, P., Carayon, P., Walker, J., & Wetterneck, T.B. (2008). Coordination in the ICU and the relation with quality of care and patient safety. Paper presented at the Healthcare Systems Ergonomics and Patient Safety (HEPS) 2008 Conference, Strasbourg, France.
Hoonakker, P., Carayon, P., Douglas, S., Schulz, K., Walker, J.M., & Wetterneck, T.B. (2008). Communication in intensive care units. Paper presented at the ODAM 2008 Conference, Guaruja, Brazil.
Paris, B., Carayon, P., Wetterneck, T.B., Blosky, M.A., & Walker, J. (2008). Safety of the antibiotic medication use process in the intensive care unit. Proceedings of the Human Factors and Ergonomics Society (HFES) 52nd Annual Meeting,New York, NY.
Wetterneck, T., Paris, B., Blosky, M.A., Cartmill, R.S., Walker, J., & Carayon, P. (2008). Error recovery in the ICU medication use process. Paper presented at the Healthcare Ergonomics Systems and Patient Safety (HEPS) 2008 Conference, Strasbourg, France.
Wetterneck, T.B., Paris, B., Walker, J., & Carayon, P. (2008). CPOE functionalities and medication ordering errors in the ICU. Proceedings of the 9th International Symposium on Human Factors in Organizational Design and Management,Sao Paulo, Brazil.
Van Roy, K.S. (2008). The perceived workload of ICU physicians and physician extenders and its relationship to burnout, patient safety, and quality care. PhD Dissertation, University of Wisconsin – Madison.
2007
Hoonakker, P. & Carayon, P. (2007). Selection of an instrument to measure end-user satisfaction. Paper presented at the Work with Computing Systems (WWCS) Conference, Stockholm, Sweden.
2006
Schultz, K., Slagle, J., Brown, R., Douglas, S., Frederick, B., Lakhani, M., Scruggs, J., Slater, B., Weinger, M.B., Wood, K.E., & Carayon, P. (2006). Development of job analysis tool for assessing the work of physicians in the intensive care unit. Paper presented at the Human Factors and Ergonomics Society (HFES) 50th Annual Meeting, Santa Monica, CA.
Li, Q., Douglas, S., Hundt, A.S., & Carayon, P. (2006). A heuristic evaluation of a computerized provider order entry (CPOE) technology. Proceedings of the IEA 2006 Congress, Elsevier Ltd.: London.
Questionnaires
3 Month Post-Implementation Questionnaire
12 Month Post-Implementation Questionnaire – Physicians
12 Month Post-Implementation Questionnaire – Nurses
Usability Resources
In this research, a participatory ergonomics model served as the basis for creating a network of usability coordinators. These coordinators work in various capacities at Geisinger Health System in Information Technology and Operations (both local and regional) related to the design and implementation of an electronic health record (EHR). Through hands-on training and practice evaluation sessions, researchers impart knowledge, oversee testing, and guide follow-up on usability issues related to user-centered design of software and the HIT interface. The knowledge and usability skills ultimately become “standard operating procedure” throughout the continuous design, redesign and implementation of the EHR. Likewise the network of usability coordinators serves as a resource for fellow coordinators as they assist and provide feedback to each other in the design, execution, evaluation, and follow up related to usability evaluations they conduct.
Training resources include:
Training Session I covers usability principles and methods. As a result of this introductory session, all participants conducted paired heuristic evaluations of one aspect of the EHR, the electronic medication administration record (eMAR). The following morning the attendees reconvened and prioritized the findings of the evaluation.
Session 1: Usability Evaluations: Heuristic Evaluations
Usability Evaluation Planning Document
Heuristic Evaluation Definitions and Worksheet
Heuristic Evaluation Script
Training Session II covers scenario-based usability evaluations. The recently-trained usability coordinators instructend-users to perform tasks. The end-users’ facility when performing various tasks, as well as the verbal assessments they shared, were collected and later presented at a wrap-up session when the usability coordinators reconvened the following day. Once again results were prioritized.
Session 2: Usability Evaluations Through Scenario-Based Evaluations
Usability Evaluation Planning Document
Specifications, Script, Data, Collection Tool for eMAR Usability Testing Evaluation
References: An annotated bibliography of journal articles, books and other references that address important concepts of usability and usability principles and results of actual tests applied to health information technology.