Epic Signal Data 2023

For more information about the report methodology and Epic Signal, scroll to the end of this page.

Best Practices & Principles for Leveraging Signal Data

Many organizations use Signal data to initiate conversations with their providers about EHR use. Signal data cannot accurately predict EHR satisfaction, burnout, or turnover. However, through the analysis of top-performing Arch Collaborative members and their efforts in conjunction with Signal data, KLAS has identified key principles for successfully utilizing Signal data to improve provider EHR satisfaction:

Use Signal to identify outliers among provider groups and determine where efficiency can be improved through optimization and training.

Through informaticists or trainers, use Signal data to identify areas requiring future training or optimization efforts.

Approach providers in a non-punitive manner to understand their EHR usage and address any pain points.

Provide training or education on best practices to enhance EHR efficiency and help users personalize their workflow.

Continue measuring progress, taking an iterative approach to improvement efforts.

Since Signal data may not identify all providers struggling with the EHR, it is important to regularly measure providers’ EHR satisfaction as well—this can be done through the Arch Collaborative’s user experience surveys or through consultation with department or physician group leaders to identify providers who need help.

Signal Helps as a Conversation Starter—but Not as a Predictor of EHR Dissatisfaction, Burnout, or Turnover

As already noted, Signal data is not a good predictor of an individual provider’s EHR satisfaction as measured by the Collaborative’s EHR Experience Survey. For example, the chart below shows that many providers whose Signal data suggests they have low satisfaction are actually very satisfied and vice versa. Part of this disconnect is rooted in the inherent differences in how the two sources of data are collected and what they measure. Epic Signal data is closely tied to operational understanding—when and how the EHR is being used, what features are being used, and the nature of provider workloads. Therefore, based on a Signal metric like how much time users spend in the system, a disengaged user may appear similar to someone who is highly efficient, making it difficult to pinpoint correlation with EHR experience. In contrast, Arch Collaborative data measures overall EHR perception and is not bound to specific time frames like Epic Signal data.

Still, Signal data can be very useful as a stepping-off point to work on the EHR experience. The high-performing (i.e., high clinician satisfaction with the EHR) Arch Collaborative members detailed below shared ways in which they use Signal data to spark conversations and guide interactions with providers. (Click to navigate to the full case studies.)

Targeting Individual User Improvement

Houston Methodist uses Signal data to identify areas for improvement in their EHR system. They seek feedback from users experiencing issues and work with operational managers to ensure access to individual physician Signal data and facilitate conversations with physicians. The organization employs scripting to approach physicians in a non-punitive manner, using available data to address their pain points and invite them to have further discussions—all with a focus on understanding and sharing best practices related to Notes, In Basket, and other tools.

WellSpan Health uses Signal data and screen recordings to improve provider efficiency. The data is used to recommend personalization tools for providers and to troubleshoot issues. The physician informatics training and support liaison team also uses the data to help providers set up the EHR according to their preferences.

Penn Medicine uses Signal data and provider surveys to tailor personalization efforts to each provider. The data helps determine what should be covered during the session, taking into account the provider's specialty, role, and specific needs.

Measuring Training Effectiveness

UCSF Health uses Epic’s Signal tool to measure proficiency at the beginning of their ongoing EHR training for established APPs. The Signal data identifies areas of opportunity for each APP, and the courses are tailored accordingly. The results of the training are also evaluated using Signal data, which shows the training results in an average 21% increase in EHR proficiency.

Franciscan Health uses Signal data to monitor EHR usage metrics and determine the success of their training programs. On an ongoing basis, the data is used to monitor provider progress and identify areas of improvement.

SUNY Upstate Medical University uses Signal data to identify areas of focus and target specific Epic functionality that can save providers time in the system. The data is also used to guide discussions and show providers ways to increase their efficiency. Training interventions are documented in Signal, allowing for the tracking of outcomes.

Monitoring & Communicating Trends

Lee Health uses Signal data to inform training and optimization efforts. The data is used to analyze specific EHR metrics, compare providers to their peers, and identify opportunities for efficiency improvement. The medical director also sends a snapshot of Signal data to the provider showing them their EHR use trends.

Advocate Aurora Health uses Signal data in their systematic approach to In Basket oversight. The data is used to provide focus for oversight efforts and monitor results in their ongoing project to ensure each notification is meaningful or actionable for the clinical end user.

With Small Statistical Correlation, a Few Signal Metrics May Be Helpful Markers for EHR Experience

In the predictive algorithm KLAS built (see Research Methodology), a few Signal variables show a small yet statistically significant effect on the EHR experience. These include the amount of time providers spend in the EHR per day during scheduled hours, related to overall EHR satisfaction (r²=0.03), and the amount of time spent on In Basket tasks per day, related to burnout (r²=0.02). Additional analysis was conducted on some of the specific metrics that make up the NEES, including ease of learning, efficiency, and specialty-specific functionality. For the first two examples, the most important Signal metric is the amount of time spent in the EHR per day during scheduled hours. For satisfaction with specialty-specific functionality, the Signal metric most correlated with success is the amount of time spent on orders per day.

Research Methodology

This report relies on information from 16 organizations who (1) have measured their clinicians’ EHR experience with the Arch Collaborative in the past three years and (2) expressed interest in understanding how their Signal data correlates with aspects of the clinician experience. Using the Boruta algorithm for variable selection, KLAS created a complex predictive model to measure correlation between Signal data and Arch Collaborative metrics. In the end, the algorithm found that 76 different Signal metrics merited inclusion in the model. KLAS mapped Signal data to our EHR Experience Survey results using first and last names, spanning over 5,500 providers. Only Signal data from the ambulatory environment was used.

What Is Epic Signal?

Signal helps you take a data-driven approach to measuring provider efficiency with Epic, both within your organization and across the Epic community. Compare groups within your organization to each other or compare your metrics to the Epic community average to help identify the most important workflows to target for efficiency improvements.

Signal shows how your providers are using Epic in four key areas—In Basket (for inpatient providers, Communications), Orders, Notes & Letters (for inpatient providers, Notes), and Clinical Review—and suggests features and configuration options to increase efficiency in each area based on your data. Suggestions are easily actionable and personalized to your organization’s needs.

Signal includes two levels of data:

• The Summary view shows organization-wide data. It helps leaders, analysts, and informaticists see larger efficiency trends across specialties and your entire healthcare system. You can choose between the EpicCare Ambulatory and EpicCare Inpatient views in the upper-right corner of the screen.
• The Provider view drills down to individual provider data. It helps trainers and others who assist providers one on one to provide personalized guidance based on system usage patterns. This view is currently available only for ambulatory providers.

Click here for a quick start guide on Galaxy. Click here to learn what’s new with Signal.

What Is the KLAS Arch Collaborative?

The Arch Collaborative is a group of healthcare organizations committed to improving the EHR experience through standardized surveys and benchmarking. To date, almost 300 healthcare organizations have surveyed their end users and over 400,000 clinicians have responded. Reports such as this one seek to synthesize the feedback from these clinicians into actionable insights that organizations can use to revolutionize patient care by unlocking the potential of the EHR.