Department of Medicine



Featured Publications


Our team has conducted research that has led to publications in several high quality professional and academic journals. Below are abstracts from our team’s featured publications; follow the links provided to read the articles.

The Path to Diagnostic Excellence Includes Feedback to Calibrate how Clinicians Think. (Meyer AND, Singh H, 2019) Improving diagnosis in health care is considered the next imperative for patient safety. Rapidly evolving diagnostic tests and treatments and competing priorities and pressures encountered by clinicians to deliver high-quality, low-cost health care make this a major challenge. Clinicians frequently balance undertesting, possibly missing a diagnosis, with pursuing overzealous diagnostic testing, which could be harmful and costly. Rigorous multidisciplinary research and innovation from cognitive psychology, human factors, informatics, and social sciences are needed to stimulate previous efforts to reduce diagnostic errors.

Application of Electronic Trigger Tools to Identify Targets for Improving Diagnostic Safety. (Murphy DR, Meyer AND, Sittig DF, et. al., 2018) Electronic trigger (e-trigger) tools, which mine vast amounts of patient data to identify signals indicative of a likely error or adverse event, offer a promising method to efficiently identify diagnostic errors. The authors present a knowledge discovery framework, the Safer Dx Trigger Tools Framework, that enables health systems to develop and implement e-trigger tools to identify and measure diagnostic errors using comprehensive electronic health record (EHR) data. The authors outline key future research, including advances in natural language processing and machine learning, needed to improve effectiveness of e-triggers. Integrating diagnostic safety e-triggers in institutional patient safety strategies can accelerate progress in reducing preventable harm from diagnostic errors.

Beyond Dr. Google: The Evidence about Consumer-Facing, Digital Tools for Diagnosis. (Millenson ML, Baldwin JL, Zipperer L, et. al., 2018) Direct-to-consumer (DTC), interactive, diagnostic apps with information personalization capabilities beyond those of static search engines are rapidly proliferating. While these apps promise faster, more convenient and more accurate information to improve diagnosis, little is known about the state of the evidence on their performance or the methods used to evaluate them. The authors conducted a scoping review of the peer-reviewed and gray literature and found that apps vary widely in functionality, accuracy, safety and effectiveness, although the usefulness of this evidence was limited by a frequent failure to provide results by named individual app.

Methods for Patient-Centered Interface Design of Test Result Display in Online Portals. (Nystrom DT, Singh H, Baldwin J, et. al., 2018) Patients have unique information needs to help them interpret and make decisions about laboratory test results they receive on web-based portals. However, current portals are not designed in a patient-centered way and little is known on how best to harness patients’ information needs to inform user-centered interface design of portals. The authors designed a patient-facing laboratory test result interface prototype based on requirement elicitation research and used a mixed-methods approach to evaluate this interface. They found that requirement elicitation studies can inform the design of a patient-facing test result interface, but considerable user-centered design efforts are necessary to create an interface that patients find useful. To promote patient engagement, health information technology designers and developers can use similar approaches to enhance user-centered software design in patient portals.

Adherence to recommended electronic health record safety practices across eight health care organizations. (Sittig DF, Salimi M, Aiyagari R, et. al., 2018) The Safety Assurance Factors for EHR Resilience (SAFER) guides were released in 2014 to help health systems conduct proactive risk assessment of electronic health record (EHR)- safety related policies, processes, procedures, and configurations. The extent to which SAFER recommendations are followed is unknown. The authors conducted risk assessments of 8 organizations of varying size, complexity, EHR, and EHR adoption maturity. They found that, despite availability of recommendations on how to improve use of EHRs, most recommendations were not fully implemented. They suggest new national policy initiatives are needed to stimulate implementation of these best practices.

Evaluating a mobile application for improving clinical laboratory test ordering and diagnosis. (Meyer AND, Thompson PJ, Khanna A, et. al., 2018) Mobile applications for improving diagnostic decision making often lack clinical evaluation. The authors evaluated if a mobile application improves generalist physicians' appropriate laboratory test ordering and diagnosis decisions and assessed if physicians perceive it as useful for learning. They found that a mobile app, PTT Advisor, may contribute to better test ordering and diagnosis, serve as a learning tool for diagnostic evaluation of certain clinical disorders, and improve patient outcomes. Similar methods could be useful for evaluating apps aimed at improving testing and diagnosis for other conditions.

Electronic health record reviews to measure diagnostic uncertainty in primary care. (Bhise V, Rajan SS, Sittig DF, et al., 2018) Diagnostic uncertainty is common in primary care. Because it is challenging to measure, there is inadequate scientific understanding of diagnostic decision-making during uncertainty. The authors' study found that, while current diagnosis coding mechanisms (ICD-9 and ICD-10) are unable to capture uncertainty, review of EHR documentation can help identify diagnostic uncertainty with moderate reliability.

Patient perceptions of receiving test results via online portals: a mixed-methods study (Giardina TD, Baldwin J, Nystrom DT, et al., 2017) Online portals provide patients with access to their test results, but it is unknown how patients use these tools to manage results and what information is available to promote understanding. A mixed-methods study was conducted to explore patients’ experiences and preferences when accessing their test results via portals. The author's findings suggest that online portals are not currently designed to present test results to patients in a meaningful way. They found that simply providing access via portals is insufficient, and additional strategies are needed to help patients interpret and manage their online test results.

Electronic Triggers to Identify Delays in Follow-Up of Mammography: Harnessing the Power of Big Data in Health Care (Murphy DR, Meyer AN, Vaghani V, et al., 2017) The authors conducted a study to evaluate the effectiveness of an electronic trigger to flag delayed follow-up on mammography. Flagged records were reviewed to determine the trigger's performance characteristics. The frequency of delays and patient communication related to abnormal results, reasons for lack of follow-up, and whether patients were subsequently diagnosed with breast cancer were also assessed. The authors concluded that clinical application of mammography-related triggers could help detect delays in follow-up of results.

An electronic trigger based on care escalation to identify preventable adverse events in hospitalised patients (Bhise V, Sittig DF, Vaghani V, et al., 2017). The authors refined the methods of Institute of Healthcare Improvement’s Global Trigger Tool (GTT) application and leveraged electronic health record (EHR) data to improve detection of preventable adverse events. The EHR data-based trigger and modified review process were able to efficiently identify hospitalized patients with preventable adverse events, including diagnostic errors. Such e-triggers can help overcome limitations of currently available methods to detect preventable harm in hospitalized patients.

The Burden of Inbox Notifications in Commercial Electronic Health Records. (Murphy DR, Meyer AND, Russo E, et. al., 2016) With wider use of electronic health records (EHRs), physicians increasingly receive notifications via EHR-based inboxes (e.g., Epic’s “In-Basket” and GE Centricity’s “Documents”). Types of notifications include test results, referral responses, medication refill requests, and messages from clinicians, among others. Information overload is of emerging concern because new types of EHR-based notifications and ‘FYI’ messages can be easily created (versus in paper-systems) and this additional workload remains uncompensated despite reimbursement reductions. Moreover, EHRs make it easier to measure information load. The authors quantified notifications physicians received via inboxes of commercial EHRs to estimate their burden.

Challenges in Patient Safety Improvement Research in the Era of Electronic Health Records. (Russo E, Sittig DF, Murphy DR, et. al., 2016) Electronic health record (EHR) data repositories contain large volumes of aggregated, longitudinal clinical data that could allow patient safety researchers to identify important safety issues and conduct comprehensive evaluations of health care delivery outcomes. However, few health systems have successfully converted this abundance of data into useful information or knowledge for safety improvement. In this paper, the authors use a case study involving a project on missed/delayed follow-up of test results to discuss real-world challenges in using EHR data for patient safety research. To leverage EHRs and their abundant data for patient safety improvement research, many current data access and security policies and procedures must be rewritten and standardized across health care organizations. These efforts are essential to help make EHRs and EHR data useful for progress in our journey to safer health care.

Computerized Triggers of Big Data to Detect Delays in Follow-up of Chest Imaging Results (Murphy DR, Meyer AN, Bhise V et al., 2016). A “trigger” algorithm was used to identify delays in follow-up of abnormal chest imaging results in a large national clinical data warehouse of electronic health record (EHR) data. The report found that the application of triggers on “big” EHR data may aid in identifying patients experiencing delays in diagnostic evaluation of chest imaging results suspicious for malignancy.

The global burden of diagnostic errors in primary care (Singh H, Schiff GD, Graber ML, 2016). The authors discuss the global significance, burden and contributory factors related to diagnostic errors in primary care. They also summarize interventions based on available data and suggest next steps to reduce the global burden of diagnostic errors. They recommend that the World Health Organization (WHO) consider bringing together primary care leaders, practicing frontline clinicians, safety experts, policymakers, the health IT community, medical education and accreditation organizations, researchers from multiple disciplines, patient advocates, and funding bodies among others, to address the many common challenges and opportunities to reduce diagnostic error.

Measuring and improving patient safety through health information technology: The Health IT Safety Framework (Singh H, Sittig DF, 2015). In response to the fundamental conceptual and methodological gaps related to both defining and measuring health IT-related patient safety, the authors propose a new framework, the Health IT Safety (HITS) measurement framework, to provide a conceptual foundation for health IT related patient safety measurement, monitoring and improvement. The framework proposes to integrate both retrospective and prospective measurement of HIT safety with an organization's existing clinical risk management and safety programs. It aims to facilitate organizational learning, comprehensive 360-degree assessment of HIT safety that includes vendor involvement, refinement of measurement tools and strategies, and shared responsibility to identify problems and implement solutions.

Improving Diagnosis in Health Care – The Next Imperative for Patient Safety (Singh H, Graber M, 2015). Based on a 1999 Institute of Medicine report, the authors discuss the history of diagnostic error, previous work and research in the area, current policies and recommendations regarding the reduction diagnostic error and future actions that can be taken to further improve diagnosis and reduce patient harm from diagnostic error.

Editorial: Helping Health Care Organizations to Define Diagnostic Errors as Missed Opportunities in Diagnosis (Singh H, 2014). An editorial discussing an article from Graber et al. In the editorial, Dr. Singh touches on challenges that healthcare organizations face regarding diagnostic errors and missed opportunities, providing a conceptual model of missed opportunities in diagnosis.

Ebola US Patient Zero: lessons on misdiagnosis and effective use of electronic health records (Upadhyay DK, Sittig DF, Singh H, 2014). The authors discuss the first travel-associated case of U.S. Ebola on Sept. 30, 2014, to highlight the public health challenge of diagnostic errors and discuss the effective use of EHRs in the diagnostic process. We analyze the case to discuss several missed opportunities and outline key challenges and opportunities facing diagnostic decision-making in EHR-enabled healthcare.

The frequency of diagnostic error in outpatient care: estimations from three large observational studies involving US adult populations (Singh H, Meyer AN, Thomas, EJ, 2014). The authors estimated the frequency of diagnostic errors in the U.S. adult population by synthesizing data from three previous studies of clinic-based populations that used conceptually similar definitions of diagnostic error. Their estimate suggests that diagnostic errors affect at least 1 in 20 U.S. adults. This foundational evidence should encourage policymakers, healthcare organizations and researchers to start measuring and reducing diagnostic errors.

Types and Origins of Diagnostic Errors in Primary Care Settings (Singh H, Giardina TD, Meyer AN et al., 2013). The authors reviewed medical records of diagnostic errors detected at two sites through EHR–based triggers. It was found that most errors were related to process breakdowns in the patient-practitioner clinical encounter. It was found that the diagnostic errors identified in the study involved a large variety of common diseases and had significant potential for harm. Most errors were related to process breakdowns in the patient-practitioner clinical encounter.

Electronic health record-based surveillance of diagnostic errors in primary care (Singh H, Giardina TD, Forjuoh SN et al., 2012). Diagnostic errors in primary care are harmful but difficult to detect. The authors tested an electronic health record (EHR)-based method to detect diagnostic errors in routine primary care practice. The report found that while physician agreement on diagnostic error remains low, an EHR-facilitated surveillance methodology could be useful for gaining insight into the origin of these errors.