What Can Epic's Signal Data Tell Us About EHR Satisfaction and Burnout?
Through the Signal tool, organizations using Epic have easy access to a wealth of user-level Physician Efficiency Profile (PEP) data, including in-depth data on how physicians use In Basket, orders, notes, and clinical review. Provider executives use this data to inform their informatics strategies and identify how users compare to peers in the Epic community. Leaders looking to improve EHR satisfaction and reduce burnout among their clinicians may wonder, Can this data be used to identify individual physicians who are struggling with the EHR?
Want to see full details?
Want to see full details?
Here is my information:
Promising Trends to Watch:
Which Signal Metrics Show Preliminary Correlations with EHR Satisfaction and Burnout?
While Signal data is not designed to be a predictor of an individual physician’s EHR satisfaction or feelings of burnout, many high-performing Epic organizations in the Arch Collaborative have found success using their Signal data to spark conversations and guide interactions with physicians (see full report for examples). Additionally, several Signal metrics do show some degree of correlation with EHR satisfaction or burnout.
The Signal metric that currently does the best at predicting EHR satisfaction and burnout is the physician’s same-day appointment-closure rate. The more appointments physicians close the same day, the higher their EHR satisfaction and the less burnout they report. Presumably, those who close more appointments per day know the tool better and are managing their workload well. However, since individual physicians’ workflows vary dramatically, using this variable as a predictor may lead to false positives (i.e., physicians who appear to be struggling but are not).
Other factors shown to impact satisfaction and burnout:
- Time spent in the EHR: Physicians with busier schedules spend more time in the EHR, get more practice with it (measured via the Signal metrics “time per day in In Basket,” “time per day in notes,” and “time per day in orders”), and thus become more proficient. These physicians also report higher levels of burnout.
- Degree of EHR personalization: Physicians that create more of their orders from preference lists and SmartSets are more likely to be satisfied with the EHR.
- Use of advanced functionality: Physicians that use SmartPhrases and Epic’s mobile applications are more likely to report high satisfaction.
- Percent of orders self-entered: Physicians who enter more of their own orders but can rely on team contributions when needed are more likely to be satisfied with the EHR.
- Time spent in the EHR on off days/hours or at home: Not surprisingly, the less time physicians spend working in the EHR on unscheduled days, during “pajama time,” and outside the hours of 7:00 AM to 7:00 PM, the less likely they are to report burnout.
Epic’s Response: Signal Changes
Epic has used these findings and other feedback from their collaboration with KLAS to make enhancements to Signal. A few specific examples include:
- Pajama time: In an effort to better quantify clinician time spent in the evenings on workdays and on days off, a new metric was created to supplement other afterhours metrics.
- Level of service and diagnosis shortcuts: Previously, Signal recorded whether level of service and diagnosis shortcuts were available to clinicians. It now records how often clinicians use them.
- Chart review filters: Previously, Signal recorded whether filters were available. It now records how often clinicians use them.
While Signal isn’t intended to be a predictor of burnout or the sole assessment of someone’s work satisfaction, it can be used in tandem with a strong clinical informatics program to identify physicians who would benefit most from additional training and support.
Signal Data Not a Reliable Predictor of EHR Satisfaction or Burnout . . . Yet
Today, only about 4% of the variation in EHR satisfaction† across individual ambulatory physicians can be predicted using Signal data (data set is limited). When we look at organization averages, this percentage increases but only to 15%. Signal data is slightly better at predicting physician burnout—5% of the variation in reported burnout between individual ambulatory physicians can be predicted using Signal data; 22% can be predicted across organizations.
†The Arch Collaborative uses the Net EHR Experience Score (NEES) to measure clinicians’ overall satisfaction with their EHR environment(s). Clinicians are asked to rate factors such as the EHR’s efficiency, functionality, impact on care, and so on, and the NEES is then calculated by subtracting the percent of negative user feedback from the percent of positive user feedback. The NEES can range from -100% (all negative feedback) to +100% (all positive feedback).
Actual vs. Predicted Net EHR Experience Score
Actual vs. Predicted Burnout Rating
Next Steps: We Need Your Help!
To date, comparing Signal usage data with EHR satisfaction data from the Arch Collaborative shows no reliable predictive correlation among ambulatory physicians. To further improve these findings, Epic and KLAS need more data points. If this research is interesting to you or your organization, please fill out this form to begin the process of adding your organization’s Signal data to our study.
Report Non-Public HTML Body
Report Public HTML Body
This material is copyrighted. Any organization gaining unauthorized access to this report will be liable to compensate KLAS for the full retail price. Please see the KLAS DATA USE POLICY for information regarding use of this report. © 2019 KLAS Research, LLC. All Rights Reserved. NOTE: Performance scores may change significantly when including newly interviewed provider organizations, especially when added to a smaller sample size like in emerging markets with a small number of live clients. The findings presented are not meant to be conclusive data for an entire client base.