EHR Optimization

Program Goals Address clinicians’ concerns regarding hardware, software, EHR documentation, and EHR education Organization Outcomes Nurses had a 24-point increase in their average Net EHR Experience Score Collaborative-Verified Best Practices Ongoing EHR Education Clinician Efficiency and Personalization Recognized Improvement Nursing EHR Success Keys to Success Simplify initial workflow training to focus on early documentation tasks and use workbooks for additional training needs Have facilitators on-site during self-guided training to answer new hires’ questions Optimize the most frequently used nurse flow sheets Ensure task force members receive recognition for their participation from organization leaders What Valley Children's Hospital Did Nurses at Valley Children’s Hospital (Valley Children’s) expressed concerns about EHR documentation and training. To address these concerns, the organization formed four task forces involving nursing leaders and bedside nurses. With strong support from nursing leadership, Valley Children’s revised initial training methods and reduced classroom training time. The organization also reduced documentation time by optimizing frequently used flow sheets, focusing on nursing areas that scored low in NEAT metrics, and reenforcing documentation by exception. Shifting the culture toward less documentation was challenging, but high-performing nurses assisted their peers during the transformation, especially with changes to flow-sheet layouts. A team approach and strong support from executives and nursing leaders were critical to success. How Valley Children's Hospital Did It

Program Goals Reduce clinician burnout related to documentation inefficiencies. Organization Outcomes Physicians using 3M™ M*Modal Fluency Direct at Baylor Scott & White Health have a Net EHR Experience Score (NEES) of 53.6, which is 7.3 points higher than the score of nonusers. Physicians using 3M™ M*Modal Fluency Direct at Baylor Scott & White Health report 14 percentage points higher agreement than nonusers that the electronic health record (EHR) enables quality care. Collaborative-Verified Best Practices Clinician Wellness and Reducing Burnout Recognized Improvement Keys to Success Involving the provider voice by getting input on changes and training around those changes. What Baylor Scott & White and 3M Did Baylor Scott & White Health initiated a medical group transformation project in 2022 to review all aspects of their organization and find ways to increase operational efficiencies on all levels. During the organization’s presentation at the KLAS Arch Collaborative Summit in 2023, Baylor Scott & White Health focused on one of the initiatives from the larger mission—to lower documentation burden among their providers. Baylor Scott & White Health has measured three times with the Arch Collaborative using the EHR experience survey since 2018, and they have seen continual improvement throughout the measurement period. Based on data captured from Epic, the average note length of Baylor Scott & White Health’s providers has more than tripled since 2009. Due to accumulating regulatory changes and clinicians who have grown accustomed to their existing ways of documenting, notes have continued to increase in length over time, which has caused patients to be confused in their reviews of the notes. Baylor Scott & White Health used two different approaches to reduce their providers’ documentation burden: re-ordering Assessment, Plan, Subjective, Objective (APSO) notes and increasing and optimizing their use of 3M™ M*Modal Fluency Direct. They got the idea of the APSO note approach from a UW Health webinar, where it was explained how UW Health initiated a project with their clinicians to flip the sequence of their note sections. This enabled UW Health’s physicians and patients to see the most crucial pieces of information at the top rather than having to scroll to the end of the notes to get to that information. After seeing what UW Health had done, the Baylor Scott & White Health team created an initiative to tackle note bloat (extraneous information commonly placed in notes that includes information contained elsewhere in the patient record) and utilized their results from the Arch Collaborative EHR experience survey to further expand the project. Note Bloat Initiative The primary goal of the note bloat initiative was to help providers to learn new documentation skills by acquiring more proficiency with the tools already available to them and to achieve higher performance standards. From the beginning, the team for the note bloat initiative involved representatives from across the organization, including individuals from the coding, compliance, and regulatory departments to ensure that any changes made wouldn’t negatively impact those downstream areas. The team designated 10–12 provider champions who represented each region of Baylor Scott & White Health and who helped develop the new note structure to pilot with primary care providers. The champions served as vocal cheerleaders for the project within their regions. The provider champions also helped gather user feedback so they could enhance the notes throughout the pilot period. During phase one of the project, Baylor Scott & White Health’s regional medical director led many town halls, delivering information and training on the use of this new APSO note type and soliciting from the 100 primary care providers piloting the project any feedback to further refine the process. One major cultural challenge the Baylor Scott & White Health team confronted as they sought to reduce the provider documentation burden was helping providers understand which pieces of information were no longer needed within each note; many aspects of their notes were already captured elsewhere in the EHR . As Baylor Scott & White Health piloted the APSO note initiative with primary care providers, they embedded a link in the EHR to track who was using the APSO notes and how the note structure was impacting their time spent documenting in the EHR. The pilot group of primary care providers was not mandated to use the APSO notes, but there were over 98,000 uses of the new APSO notes in primary care as of August 2023 . The organization is in the process of developing tracking tools that are inclusive to specialist uses of the APSO notes, so those numbers have not yet been reported. Prior to the APSO note initiative, Baylor Scott & White Health was in the 75th percentile of Epic users for note length; after the initiative, the organization is now below Epic’s average for note length. Baylor Scott & White Health has noted the following phase-one outcomes : 8 fewer minutes of pajama time per day per provider. 21% shorter progress notes on average for participating providers. 7% reduction in time in notes per note per provider. 34 fewer seconds spent in notes per provider per appointment. 2 fewer minutes spent in notes per provider per day (those who did not adopt the APSO notes into their workflows saw an increase of 4.5 minutes spent in notes per provider per day on average). 166.6 estimated minutes saved writing notes per provider per month. 16,327 estimated total provider minutes saved per month. 272.1 estimated total provider hours saved per month. Increasing use of 3M™ M*Modal Fluency Direct Baylor Scott & White Health recognized that a new note type alone wouldn’t achieve the desired results for end-user satisfaction or accomplish the mission to reduce the provider documentation burden. To this end, the organization collaborated with 3M to train their clinicians more deeply on how to optimally utilize 3M™ M*Modal Fluency Direct, a front-end speech recognition solution that was already available to their clinicians. 3M Fluency Direct was embedded within Baylor Scott & White Health’s EHR system, but too few providers were using it . 3M sent an in-house solution advisor whose sole role was to augment the adoption of 3M Fluency Direct and work closely with Baylor Scott & White Health’s Epic Efficiency Program (EEP) team members, a team of analyst trainers who work on EHR proficiency and efficiency skills with providers. 3M’s solution advisor and Baylor Scott & White Health’s EEP trainers conducted scheduled sessions with providers to assess their current workflows and to teach them how to best utilize 3M Fluency Direct. Baylor Scott & White Health’s EEP trainers consist of a team of seven employees designated to improve the work lives of providers at their health system by leveraging best-practice uses of Epic’s products. Interfacing with EEP leaders enabled 3M to scale education and training to optimize the use of technology across an organization of Baylor Scott & White Health’s size. This level of engagement was critical to the success of the 3M solution advisor charged with driving better adoption, EHR experiences, and clinician satisfaction. When the 3M solution advisor reviewed Baylor Scott & White Health’s data after the advisor’s visit and training sessions, Baylor Scott & White Health saw a significant increase in 3M Fluency Direct users and an increase in the volume of minutes per month of use. The 3M solution advisor credited the one-on-one sessions and word-of-mouth promotion as the reasons for higher adoption and usage of 3M Fluency Direct. KLAS’ Arch Collaborative conducted an analysis of providers using 3M Fluency Direct and of those not using it in Baylor Scott & White Health’s 2023 EHR experience survey. Those using 3M Fluency Direct achieved a higher overall NEES and reported decreased after-hours charting. The group leveraging 3M’s speech understanding solution also agreed more often (than those not using the 3M technology) that the EHR enabled the delivery of quality patient care. 3M has a well-defined path to incrementally delivering artificial intelligence–powered, innovative solutions with an eye for responsible change management, starting with real-time speech understanding to proactive computer-assisted physician documentation (CAPD) and conversational virtual assistant and ambient clinical documentation. With Baylor Scott & White Health, 3M accomplished the following key project deliverables: Redesigned governance of the deployment and optimization of 3M Fluency Direct, namely through engaging with EEP leaders. Deployed a prescriptive technology package. Planned the 3M Fluency Direct adoption expansion, strategic onboarding, and a scale-up plan alongside the EEP team’s educational efforts. Applied 3M Fluency Direct efficiencies to efforts around medical group documentation transformation. Physician endorsement of 3M Fluency Direct  One physician specifically enthused by their use of 3M Fluency Direct created a video of them completing a Medicare annual wellness note within 60 seconds using natural speech, demonstrating to their peers what could be achieved with 3M Fluency Direct. In the short video, the physician explained how they completed visit notes using Epic smart tools with 3M Fluency Direct and finished their notes within the same day of the visit. The physician covered four achievements through their process: Increased use of Epic smart tools to become more efficient with documentation. No violations of the “OHIO Principle” ( “only handle it once”), because the more a physician goes in and out of office visits without completing documentation, the longer it will take them to get it all done. Noticeable improvement in patient satisfaction since the physician was on time more often with their appointments. Increased personal EHR satisfaction by completing charts during the same day the visit occurred. During the video, the physician walked viewers through a real visit where the physician had pre-charted things and then, while in the room, completed their wrap-up documentation, decided on the correct billing codes, and placed information in the after-visit summary. The physician accomplished everything conversationally using 3M Fluency Direct. This enthused physician encouraged other interested providers to reach out to the right people at Baylor Scott & White Health to better understand what efficiencies are possible to improve their own documentation. Such endorsements by organizational and clinical leaders can help meaningfully drive interest among clinicians in the use of new and existing informatics solutions.

Program Goals Improve clinicians’ satisfaction with the EHR Have clinicians and departments buy in to the concept that they have responsibility for their EHR use and satisfaction Organization Outcomes 28-point improvement in Net EHR Experience Score 24-point improvement in agreement that the EHR enables efficiency 17-point improvement in agreement that the EHR is needed functionality Collaborative-Verified Best Practices Ongoing EHR Education Recognized Improvement Keys to Success Build a network of departmental support and provide clinicians with clear instructions on how to be successful with the EHR Invest in ongoing EHR education. Many issues that clinicians think might require a fix in the EHR can typically be resolved with more education What Amsterdam UMC Did After receiving their initial Arch Collaborative survey results in 2018, Amsterdam UMC chose to make significant adjustments to change the perception and utility of their EHR. They implemented three key programs to begin this transformation. First, they revamped their service ticket process to better meet the needs of clinicians regarding their EHR use. By triaging all incoming tickets, they determined integrality, clarity, and priority before sending the tickets to an application team to work on. Second, they changed strategies on their governance structure and positioned a strong CMIO, CNIO, and CSIO to work closely with Amsterdam UMC’s EHR department and to implement an ambassador program where every department was expected to provide a clinical ambassador who would spend approximately 10 hours per month in ambassador activities. Third, they created an ongoing, one-on-one EHR education program. How Amsterdam UMC Did It

Program Goals Foster an environment where providers are valued and feel they are making a difference Organization Outcomes Providers at 100th percentile for experiencing no symptoms of burnout Physicians at 90th percentile for not being likely to leave within two years Collaborative-Verified Best Practices Clinician Wellness Keys to Success Employ a thorough vetting process for potential new providers to ensure they are good fit for the organization and are prepared to work with people experiencing homelessness Provide support to new providers through frequent one-on-ones with medical directors where the providers can work through challenges Allow providers to provide patient-centered care by encouraging them to build rapport and by streamlining their documentation process Facilitate a community culture within the organization by celebrating individuals and sharing successes What Circle the City Did Circle the City, a healthcare organization that focuses on serving individuals experiencing homelessness, has achieved remarkable success in reducing burnout and improving provider satisfaction. The leaders at Circle the City emphasize the importance of aligning the organization's mission with its recruitment process, building a strong support structure, and fostering a sense of community among the providers. The organization’s approach highlights the significance of creating a fulfilling work environment and ensuring that the providers feel supported and valued. How Circle the City Did It

For more information about the report methodology and Epic Signal, scroll to the end of this page. Best Practices & Principles for Leveraging Signal Data Many organizations use Signal data to initiate conversations with their providers about EHR use. Signal data cannot accurately predict EHR satisfaction, burnout, or turnover. However, through the analysis of top-performing Arch Collaborative members and their efforts in conjunction with Signal data, KLAS has identified key principles for successfully utilizing Signal data to improve provider EHR satisfaction: Use Signal to identify outliers among provider groups and determine where efficiency can be improved through optimization and training. Through informaticists or trainers, use Signal data to identify areas requiring future training or optimization efforts. Approach providers in a non-punitive manner to understand their EHR usage and address any pain points. Provide training or education on best practices to enhance EHR efficiency and help users personalize their workflow. Continue measuring progress, taking an iterative approach to improvement efforts. Since Signal data may not identify all providers struggling with the EHR, it is important to regularly measure providers’ EHR satisfaction as well—this can be done through the Arch Collaborative’s user experience surveys or through consultation with department or physician group leaders to identify providers who need help. Signal Helps as a Conversation Starter—but Not as a Predictor of EHR Dissatisfaction, Burnout, or Turnover As already noted, Signal data is not a good predictor of an individual provider’s EHR satisfaction as measured by the Collaborative’s EHR Experience Survey. For example, the chart below shows that many providers whose Signal data suggests they have low satisfaction are actually very satisfied and vice versa. Part of this disconnect is rooted in the inherent differences in how the two sources of data are collected and what they measure. Epic Signal data is closely tied to operational understanding—when and how the EHR is being used, what features are being used, and the nature of provider workloads. Therefore, based on a Signal metric like how much time users spend in the system, a disengaged user may appear similar to someone who is highly efficient, making it difficult to pinpoint correlation with EHR experience. In contrast, Arch Collaborative data measures overall EHR perception and is not bound to specific time frames like Epic Signal data. Still, Signal data can be very useful as a stepping-off point to work on the EHR experience. The high-performing (i.e., high clinician satisfaction with the EHR) Arch Collaborative members detailed below shared ways in which they use Signal data to spark conversations and guide interactions with providers. (Click to navigate to the full case studies.) Targeting Individual User Improvement Houston Methodist uses Signal data to identify areas for improvement in their EHR system. They seek feedback from users experiencing issues and work with operational managers to ensure access to individual physician Signal data and facilitate conversations with physicians. The organization employs scripting to approach physicians in a non-punitive manner, using available data to address their pain points and invite them to have further discussions—all with a focus on understanding and sharing best practices related to Notes, In Basket, and other tools. WellSpan Health uses Signal data and screen recordings to improve provider efficiency. The data is used to recommend personalization tools for providers and to troubleshoot issues. The physician informatics training and support liaison team also uses the data to help providers set up the EHR according to their preferences. Penn Medicine uses Signal data and provider surveys to tailor personalization efforts to each provider. The data helps determine what should be covered during the session, taking into account the provider's specialty, role, and specific needs. Measuring Training Effectiveness UCSF Health uses Epic’s Signal tool to measure proficiency at the beginning of their ongoing EHR training for established APPs. The Signal data identifies areas of opportunity for each APP, and the courses are tailored accordingly. The results of the training are also evaluated using Signal data, which shows the training results in an average 21% increase in EHR proficiency. Franciscan Health uses Signal data to monitor EHR usage metrics and determine the success of their training programs. On an ongoing basis, the data is used to monitor provider progress and identify areas of improvement. SUNY Upstate Medical University uses Signal data to identify areas of focus and target specific Epic functionality that can save providers time in the system. The data is also used to guide discussions and show providers ways to increase their efficiency. Training interventions are documented in Signal, allowing for the tracking of outcomes. Monitoring & Communicating Trends Lee Health uses Signal data to inform training and optimization efforts. The data is used to analyze specific EHR metrics, compare providers to their peers, and identify opportunities for efficiency improvement. The medical director also sends a snapshot of Signal data to the provider showing them their EHR use trends. Advocate Aurora Health uses Signal data in their systematic approach to In Basket oversight. The data is used to provide focus for oversight efforts and monitor results in their ongoing project to ensure each notification is meaningful or actionable for the clinical end user. With Small Statistical Correlation, a Few Signal Metrics May Be Helpful Markers for EHR Experience In the predictive algorithm KLAS built (see Research Methodology), a few Signal variables show a small yet statistically significant effect on the EHR experience. These include the amount of time providers spend in the EHR per day during scheduled hours, related to overall EHR satisfaction (r 2 =0.03), and the amount of time spent on In Basket tasks per day, related to burnout (r 2 =0.02). Additional analysis was conducted on some of the specific metrics that make up the NEES, including ease of learning, efficiency, and specialty-specific functionality. For the first two examples, the most important Signal metric is the amount of time spent in the EHR per day during scheduled hours. For satisfaction with specialty-specific functionality, the Signal metric most correlated with success is the amount of time spent on orders per day. Research Methodology This report relies on information from 16 organizations who (1) have measured their clinicians’ EHR experience with the Arch Collaborative in the past three years and (2) expressed interest in understanding how their Signal data correlates with aspects of the clinician experience. Using the Boruta algorithm for variable selection, KLAS created a complex predictive model to measure correlation between Signal data and Arch Collaborative metrics. In the end, the algorithm found that 76 different Signal metrics merited inclusion in the model. KLAS mapped Signal data to our EHR Experience Survey results using first and last names, spanning over 5,500 providers. Only Signal data from the ambulatory environment was used. What Is Epic Signal? Information provided by Epic Signal helps you take a data-driven approach to measuring provider efficiency with Epic, both within your organization and across the Epic community. Compare groups within your organization to each other or compare your metrics to the Epic community average to help identify the most important workflows to target for efficiency improvements. Signal shows how your providers are using Epic in four key areas—In Basket (for inpatient providers, Communications), Orders, Notes & Letters (for inpatient providers, Notes), and Clinical Review—and suggests features and configuration options to increase efficiency in each area based on your data. Suggestions are easily actionable and personalized to your organization’s needs. Signal includes two levels of data: The Summary view shows organization-wide data. It helps leaders, analysts, and informaticists see larger efficiency trends across specialties and your entire healthcare system. You can choose between the EpicCare Ambulatory and EpicCare Inpatient views in the upper-right corner of the screen. The Provider view drills down to individual provider data. It helps trainers and others who assist providers one on one to provide personalized guidance based on system usage patterns. This view is currently available only for ambulatory providers. Click here for a quick start guide on Galaxy. Click here to learn what’s new with Signal. What Is the KLAS Arch Collaborative? The Arch Collaborative is a group of healthcare organizations committed to improving the EHR experience through standardized surveys and benchmarking. To date, almost 300 healthcare organizations have surveyed their end users and over 400,000 clinicians have responded. Reports such as this one seek to synthesize the feedback from these clinicians into actionable insights that organizations can use to revolutionize patient care by unlocking the potential of the EHR.

Methodology For the purposes of this report, “eLearning” refers to self-directed electronic learning. “Virtual training” refers to the broader combination of eLearning and instructor-led virtual training. The data in this report comes from four sources † : (1) clinician responses to the EHR Experience Survey, (2) clinician responses to the Trainer Quality Benchmark Survey, (3) responses from healthcare organization leaders to the Executive Survey, and (4) interviews with organizations that receive high marks from clinicians for the helpfulness and effectiveness of tip sheets and self-directed online training. † About KLAS Arch Collaborative surveys: The EHR Experience Survey asks clinicians approximately 40 questions about their EHR experience and how it relates to their well-being and ability to care for patients. Key metrics from this survey are used to create an overall Net EHR Experience Score and to generate peer benchmarking. The Trainer Quality Benchmark Survey asks clinicians 11 questions about their satisfaction with the EHR training they have received and the trainer who provided it. This data allows organizations to compare their training with that of other organizations and allows them to benchmark satisfaction across individual trainers at their organization. The Executive Survey is designed to provide perspective on EHR governance and utilization and includes approximately 27 questions related to training and education efforts. eLearning Done Right Can Be Just as Impactful as Other Training Modalities Many healthcare organizations are deploying eLearning tools that help them successfully scale education, enable clinicians to learn at their own pace, and free up training resources to spend more time both creating valuable specialty- and role-specific materials and interacting with clinicians individually. While past Arch Collaborative research (such as the 2021 Clinician Training report) showed in-person training was significantly more effective than virtual learning, additional data collected in recent years shows that robust eLearning can have an equally meaningful impact on the clinician experience. In-person training, while still associated with the highest satisfaction, is only slightly ahead of eLearning. When eLearning is done well, it doesn’t come with a sacrifice in impact. This trend holds true across clinical backgrounds, organization types, and EHR solutions. And over the last few years, clinician satisfaction with virtual training ‡ has increased. ‡ Virtual training includes both instructor-led virtual training (such as webinars or remote classrooms) and self-directed eLearning. The KLAS Arch Collaborative first began measuring virtual training as a single category in 2020 but has recently begun to collect data on the two distinct categories. Future reports will continue to explore the differences between instructor-led virtual training and self-directed eLearning. Strong eLearning Is Connected to Lower Clinician Burnout and Higher Trust in IT Leaders  Arch Collaborative data shows that increased adoption of virtual training is correlated with a significantly better clinician experience. The quality of training is also vital to maximizing the impact—clinicians who agree virtual EHR education is helpful and effective are less likely to report feeling burned out. These clinicians are also 73 times more likely to report satisfaction with organization and IT leadership around the EHR (compared to those strongly dissatisfied with EHR education). Some of this effect is rooted in inherent benefits of the eLearning method—eLearning is usually easier to access and consume, can be tailored to specific roles, and is often packaged in shorter sessions. eLearning tools are also continuing to evolve and incorporate sophisticated adult learning methods, which may further increase eLearning’s impact on the clinician experience. Clinicians’ Sense of Value from eLearning Is Trending Upward While clinicians have historically reported at-the-elbow EHR training is their preferred training type, an increasing number are embracing the advantages of eLearning. Data from the Trainer Quality Benchmark Survey shows the percentage of clinicians who would recommend eLearning courses to a colleague has increased from 67% in 2021 to 83% in 2023. This coincides with an increase in the percentage of clinicians who agree eLearning courses were very valuable—64% in 2021 compared to 80% in 2023. The quality of eLearning programs and platforms has improved, and newer entrants to medical practice are more likely to have used these same types of platforms during their education. All of this means clinicians are more likely to engage with education in an eLearning format. Many clinicians say they enjoy the self-paced and on-demand nature of eLearning tools. Having access to a library of learning modules empowers clinicians to review concepts as needed to reinforce previous learning and learn new workflows that improve their EHR efficiency. Voice of the Clinician “I prefer to do virtual self-directed training. I can review whatever I might need to. I like direct contact with a person, but I don’t remember all they say, and then they need to repeat what they have said. I feel it’s not as efficient as when I’m working with the computer on my own.” —Nurse “Self-directed training gives me autonomy to go through the training at my own speed and really learn what steps I am taking.” —Physician eLearning Improves Scalability and Reduces Expenses Associated with EHR Education As healthcare organizations grow, scalability of training programs is a top concern. Frequent updates and changes to the EHR can stretch thin the resources allocated to training and education. One of eLearning’s biggest advantages is scalability—both in content and total number of learners. For example, eLearning modules can be modified or added to at any time, and the most up-to-date materials can be automatically distributed to the relevant clinicians. Another unique characteristic of eLearning is that it provides consistency—content is created at a single point and deployed so that every learner receives the exact same information. This can reduce challenges associated with trainer variation, and it can ensure every clinician receives the highest-quality content possible. eLearning is especially helpful in empowering clinicians to consume education in their own clinic/hospital, without the need for clinicians or trainers to spend precious time traveling. eLearning also enables organizations to quickly customize and distribute highly specialized materials to the right subset of learners. Learn from Your Peers: Allina Health Allina Health had a large EHR upgrade that affected the workflows of 9,000 employees in more than 50 roles. Because approximately 80% of the content was the same for all roles, the LCMS (learning content management system) allowed Allina to create one base eLearning and then quickly customize the other 20% of content. This resulted in 11 unique eLearnings, and employees received education that was customized to their role. Without the LCMS, these 11 eLearnings would have taken several months and significant resources to create. Instead, after the initial eLearning was complete, the other eLearnings were complete within two weeks. Increased utilization of eLearning content can also reduce costs associated with classroom-based training. Due to its web-based, asynchronous nature, eLearning is available to clinicians regardless of where they are located or when they choose to access the content. Increased utilization of eLearning reduces the overall time clinicians spend in the classroom and can generate a significant ROI in time savings. On average, clinicians who participate in self-directed eLearning report they save 90 minutes per week in the EHR for every hour they spend engaged in eLearning. Such time savings are applicable for both onboarding and ongoing training. Some organizations in the Arch Collaborative have said eLearning frees up both trainers and members of the informatics team so they can focus on other training-related tasks—e.g., content creation, at-the-elbow support. Learn from Your Peers: Baptist Health (Jacksonville) Baptist Health (Jacksonville) partnered with Divurgent to develop eLearning modules that continued the onboarding training process—successfully reducing the traditional 4-12 hours of classroom training to 1-2 hours of eLearning. Divurgent assisted Baptist Health in developing a series of simulation-based eLearning training modules. These modules were embedded in the learning management system and allowed end users to follow along and interact with the tool to properly document the case. Utilization of these eLearning modules led to a more than 11-point increase in clinicians’ satisfaction with initial training. Baptist Health also estimates these eLearning modules save $37,500 for every 25 newly onboarded physicians, while simultaneously freeing up 0.5 FTEs on the clinical informatics team. eLearning Best Practices To better understand how to effectively leverage eLearning content, we identified organizations whose EHR Experience Survey respondents reported high agreement that their organization’s tip sheets and self-directed online trainings are helpful and effective. Education leaders involved with the creation and deployment of eLearning content at four of the top ten highest scoring organizations were interviewed. The following leading practices are used by at least three of the four interviewed organizations. These leading practices are further supported by previously published Arch Collaborative case studies and data collected through Arch Collaborative surveys. Creating Effective Content Create modules that are brief (less than two hours) and interactive Carefully curate content to meet specific needs When possible, use content from EHR or third-party vendors as your foundation Expensive software is not needed to create high-quality eLearning content Optimizing Deployment Increase ease of access by using multiple, on-demand sources Provide clinicians with orientation to the eLearning platform to increase engagement, set expectations, and reinforce how to access support Ensuring Mastery Leverage proficiency assessments to track and drive individual EHR mastery Supplement eLearning content with in-person/at-the-elbow support to reinforce learning and quickly address unique needs Creating Effective Content Create modules that are brief and interactive All four of the high-performing organizations interviewed for best practices indicate that their individual eLearning modules last roughly one hour. This seems to be a common practice across organizations as 80% of clinicians say the eLearning modules they complete typically last 60 minutes or less. In fact, clinician perceptions of a training’s value decrease as the training extends beyond two hours. Keeping content concise prevents redundancy and keeps clinicians engaged for the duration. In comments from the Collaborative’s EHR Experience Survey, clinicians frequently note appreciation for the ability to easily access needed content without having to sit through lengthy classroom sessions that interfere with clinical responsibilities. In addition to being brief, eLearning content must be interactive. It is not sufficient to simply record traditional classroom content for clinicians to watch at a later time. Curriculum should be adapted for a digital format and include a hook to grab the learner’s attention, incorporate variety in the presentation, and contain knowledge checks that require the learner to engage with the material. For example, problem-based content encourages clinicians to discover how the content can directly benefit their workflows and proactively empowers them to overcome obstacles to achieving personal EHR efficiency. Voice of the Clinician “Self-directed learning is most helpful for me because I do not like adding additional meetings during the day. So being able to fit short learning videos into my schedule when I have time is helpful.” —Physician assistant “I enjoy being given direction on where to find areas of documentation and being able to click through the EHR to practice while completing training videos at my own pace. I have never felt this level of interactivity in a classroom, and it helps me focus on the content.” —Nurse Carefully curate content to meet specific needs Not all eLearning content is intended to serve the same purpose. When developing eLearning materials, identify the specific goals of the training and let those goals guide the content creation. In general, longer-form videos are more effective for initial training, while self-serve tip sheets and resources are more effective for ongoing education and in-the-moment support. For example, several organizations in the Collaborative use initial training videos to help newly hired clinicians familiarize themselves with the basics of accessing and navigating the EHR before their first day on the job so that they are better prepared to learn specialty-specific workflows once they start. Conversely, self-serve tip sheets are an effective option for ensuring clinicians adhere to standard workflows and documentation for infrequent procedures. In all cases, eLearning content should be continuously evaluated and modified based on end-user feedback to ensure it meets specific clinician needs. Case Study: Aspirus Health Aspirus Health developed a curriculum review board for all of their education content. This review board included individuals from operations and IT as well as managers and superusers. The review board was tasked with evaluating content and articulating the “why” whenever they moved something to the eLearning system. They also determined whether content would be best delivered as a video, tip sheet, or hands-on experience. By curating their educational content through uPerform, Aspirus was able to increase the effectiveness of their eLearning and drive improved engagement with both eLearning content and hands-on educational experiences. When possible, use content from EHR or third-party vendors as your foundation It is not necessary to recreate the wheel when developing eLearning content. All of the organizations interviewed for this report indicate that their EHR vendor provides content for training clinicians on EHR upgrades and workflows. Such content can be used to quickly get clinicians up to speed on EHR changes. Third-party education vendors may also have material that can be used directly or adapted as needed. By building on existing training foundations, organizations can quickly tailor content to meet their specific needs while minimizing development investment. Get Help Developing eLearning Content Check out this recent report from the Arch Collaborative for information on vendors and firms that have been validated to assist with the development of eLearning content. Expensive software is not needed to create high-quality eLearning content A common misconception is that expensive software is necessary to develop high-quality eLearning content. However, content can be created with common, inexpensive tools such as PowerPoint, Snagit, and Camtasia. As mentioned previously, eLearning modules should be concise and carefully curated. The actual tool used to convey the content is less important than the time invested in creating quality materials. Trainers and informaticists who develop eLearning modules should be trained to create content that is engaging regardless of the tool being used. Optimizing Deployment Increase ease of access by using multiple, on-demand sources All four of the interviewed high-performing organizations give their clinicians at least three different ways to access eLearning content. Common access points include learning management software, information embedded directly in the EHR dashboard, and collaborative platforms, such as Microsoft SharePoint. Having a range of methods to access on-demand eLearning content enables clinicians to both receive quick support in the moment and review longer learning modules at their own pace. Case Study: Baylor Scott & White Health Baylor Scott & White Health works together with uPerform to gather feedback from end users and then develop content that is released quarterly with information about upgrades and enhancements. When clinicians are in the middle of a patient encounter, they can use hot keys in Epic to help them understand how to complete tasks and swiftly return to patient care. In this manner, clinicians can easily access resource hubs and learning libraries and search for role-based support in uPerform. As a result of these eLearning tools, uPerform users at Baylor Scott & White Health rate their ongoing EHR training 11 percentage points higher than non-users and achieve Net EHR Experience Scores 12.9 points higher. Case Study: Franciscan Health Franciscan Health uses a dashboard to give clinicians access to all upgrade training resources, including tip sheets, PowerPoint presentations, and microlearning videos. Materials are provided by the EHR vendor and supplemented with original material from Franciscan Health. Each user can personalize the dashboard based on how much information they want to see and to show role-specific components. This strategy proactively ensures that educational materials are always conveniently accessible when EHR upgrades and changes are deployed. Provide clinicians with orientation to the eLearning platform to increase engagement, set expectations, and reinforce how to access support Organizations say resistance from clinicians who prefer classroom training is one of the biggest challenges to effective eLearning deployment. While it would be hard to completely eliminate this resistance, providing eLearning orientation can reduce the friction clinicians feel when engaging. Clinician respondents to the Arch Collaborative’s EHR Experience Survey are more likely to engage with eLearning if they are shown how to access and use the content. Though it seems simple, providing even a 15-minute orientation to introduce eLearning content, its uses, and the associated expectations can dramatically increase clinician willingness to participate because it provides them with a framework for success. Such orientation may take the form of a video or brief in-person coaching. Additionally, providing contact information for relevant support resources within the eLearning modules can create a positive experience for clinicians who encounter obstacles during training. Ensuring Mastery Leverage proficiency assessments to track and drive individual EHR mastery One difference between eLearning and in-person training is that eLearning often encourages a greater level of interaction and engagement while simultaneously promoting the use of critical thinking. By actively engaging with the EHR through hands-on eLearning modules, clinicians can achieve greater retention. Additionally, organizations can leverage eLearning tools to deliver assessments or quizzes that track EHR proficiency. eLearning can then be used to provide targeted remediation as needed. Through a combination of active engagement and proficiency assessments, eLearning modules can act as powerful drivers of EHR mastery. Case Study: Providence Providence partnered with Amplifire to enhance the effectiveness of their eLearning modules. Amplifire shows learners an explanation as many times as is needed to achieve mastery. The adaptive learning modules recognize individual struggle areas and focus on those topics to produce a learning path that leads to true proficiency. Amplifire also offers analytics that are not available in traditional classroom settings, aimed at helping identify individual knowledge gaps so they can be remediated. These tools allow Providence to see what content is resonating with learners and what isn’t so the eLearning modules can be continually evaluated and improved. Providence estimates they have achieved a 25% reduction in total training time by using adaptive learning and analytics, all while improving the quality of training. Supplement eLearning content with in-person/at-the-elbow support to reinforce learning and quickly address unique needs All four of the organizations interviewed for this report indicated that while they funnel clinicians primarily to self-serve eLearning modules, they also provide in-person and at-the-elbow support to quickly address unique user needs. One of the organizations notifies their clinical informaticists of all the newly hired clinicians so that the informaticists can proactively round and provide at-the-elbow support during these individuals’ first few weeks on the job. Another approach used is to provide contact information for support hotlines within all self-serve eLearning materials and videos so that clinicians don’t need to wait for an informaticist to come to them before receiving help. Regardless of the specific approach, all of the organizations ensure there are clear lines of communication to support teams. This is a critical element of eLearning because it prevents end users from remaining stuck for extended periods of time. By blending eLearning content with in-person support, clinicians can benefit from both the convenience and flexibility of eLearning materials as well as the knowledge and expertise provided by clinical informaticists and other support individuals. In this way, targeted in-person support can reinforce what is learned through eLearning modules and empower clinicians to improve EHR proficiency. What Is the KLAS Arch Collaborative? The Arch Collaborative is a group of healthcare organizations committed to improving the EHR experience through standardized surveys and benchmarking. To date, almost 300 healthcare organizations have surveyed their end users and over 400,000 clinicians have responded. Reports such as this one seek to synthesize the feedback from these clinicians into actionable insights that organizations can use to revolutionize patient care by unlocking the potential of the EHR.

† Each individual clinician’s responses to the Arch Collaborative EHR Experience Survey regarding core factors such as the EHR’s efficiency, functionality, impact on care, and so on are aggregated into an overall Net EHR Experience Score (NEES), which represents a snapshot of the clinician’s overall satisfaction with the EHR environment at their organization. A NEES can range from -100 (all negative feedback) to 100 (all positive feedback). † For all subjects, stages 3 and 4 are in progress. KLAS has published various  case studies , which can be found in the Arch Collaborative Learning Center. ‡ Firm & Vendor Capabilities for Clinician EHR Efficiency 2023 § EHR Education Vendor & Firm Capabilities 2023 ¶ EHR Education Software and Services 2023 How-To Guide for Using This Report Identify what clinician EHR efficiency opportunities exist at your organization Align priorities to identify the top problem to solve Read the comprehensive charts for services offerings and software solutions to see what vendors and firms can offer Go to the Vendor and Firm Insights section to see a more complete picture of each vendor’s and firm’s offerings Note: The validations in this report are not exhaustive; lack of KLAS validations in certain areas does not mean that vendors and firms don’t provide offerings in those areas. Framework of Services Offerings & Software Solutions for Clinician EHR Efficiency Services Offerings Clinical transformation strategy:  Aiding in clinician efficiency strategy/program development, informatics program development, delivery-model changes (i.e., virtual staff or top-of-license workflow adjustments), and/or project management for clinician efficiency projects. Technical build assistance:  Building, modifying, or customizing an EHR or third-party application and reconfiguring its workflow. Virtual scribes:  Providing virtual scribing services. Workflow assessment and refinement services:  Providing staff and/or project management for personalization initiatives, assessing and refining alerts/processes or configuration of messages/tasks, and creating documentation improvements using efficiency data analysis (Signal, Lights On, etc.). Vendor selection:  Helping with EHR selection, assessing EHR modules versus best-of-breed solutions, and assessing bolt-ons specifically for clinician efficiency. Interoperability optimization:  Making patient information that is received from external sources more usable for clinicians. Software Solutions Documentation burden reduction:  Aggregating, visualizing, and giving context and relevancy to EHR chart-review data; also, offering ambient voice (system listens to the patient-physician conversation and automatically documents it in the EHR) and speech recognition (real-time voice-to-text conversion), facilitating ongoing interactions and training for documentation improvement, and/or helping organize or create a relevant, contextual problem list. Message/task management (inbox):  Assessing the flow of messages/tasks to identify inefficiencies in the organization’s workflow and/or assessing message configuration to identify more efficient rules. Team communication and coordination:  Streamlining communication with members of the patient care team, automating task assignments and communication around task completion, and/or automatically sending and escalating critical lab results, imaging results, etc. Validated Services Validated Software Solutions About This Report Each year, KLAS interviews thousands of healthcare professionals about the IT solutions and services their organizations use. For this Arch Collaborative report, KLAS invited vendors and firms to provide a list of deep adopters using their clinician efficiency offerings. KLAS then interviewed these deep-adopting customers to validate their vendor’s or firm’s offering and to ask the following question: What would you tell a peer or friend about your vendor/firm? What Is the KLAS Arch Collaborative? The Arch Collaborative is a group of healthcare organizations committed to improving the EHR experience through standardized surveys and benchmarking. To date, almost 300 healthcare organizations have surveyed their end users and over 440,000 clinicians have responded. Reports such as this one seek to synthesize the feedback from these clinicians into actionable insights that organizations can use to revolutionize patient care by unlocking the potential of the EHR.

Program Goals Understand and improve EHR efficiency for clinicians Organization Outcomes Providers 89th percentile for agreement that the EHR improves efficiency Nurses 89th percentile for agreement that the EHR improves efficiency Collaborative-Verified Best Practices Clinician Efficiency and Personalization Keys to Success Monitor EHR data to identify potential EHR efficiency opportunities Proactively solicit feedback from end users through multiple channels to understand and address issues with the EHR What Houston Methodist Did Houston Methodist utilizes Signal and Nursing Efficiency Assessment Tool (NEAT) data to identify and address potential efficiency opportunities in the EHR. The organization values clinician feedback, keeping the threshold to provide feedback low with direct access to the CMIO and CNIO, and prioritizes addressing workflow concerns across disciplines. The organization openly seeks ideas from physicians and nurses on how to improve EHR efficiency, assigns informaticists to each acute care site, and provides personalized assistance and guidance for optimizing workflows. The informaticists engage with end users through various channels and aim to understand and respond to their concerns. Houston Methodist also emphasizes nursing efficiency, with an informatics-led subject matter expert group and nursing informatics representation in leadership councils. The organization analyzes NEAT data to improve flow sheets and has implemented virtual nursing for admission and discharge documentation, reducing the nursing documentation burden and improving efficiency. Houston Methodist also focuses on efficiency and workflow improvement in the ambulatory setting, constantly enhancing workflows based on user input. The Thrive team within informatics works closely with operational managers and employs scripting to approach physicians in a nonpunitive manner, using data to address their biggest opportunities to save time in the EHR How Houston Methodist Did It

Additionally, KLAS hosted table-top discussions to define three industry terms—governance, clinical informatics, and superusers—and create consistent messaging for these terms. To prepare for these discussions, Dr. Joel Gordon, Melissa Michener, Kate Bakich, Dr. Paula Scariati, and KLAS leaders met to talk about these terms and formulate discussion questions. The following report is a summary of the determined definitions as well as feedback from summit attendees. Note: Outside of the table-top discussions, summit participants were also asked various polling questions. Discussion questions and polling questions, while often similar, were asked separately and resulted in different feedback from respondents. Governance As defined by summit participants, EHR governance is the strategic and operational framework established within a healthcare organization to oversee management, use, and optimization of the EHR. Organizations’ governance strategies should involve the following elements: Decision-making: Establish a structure for making decisions related to the EHR (e.g., system selection, implementation, configuration, ongoing maintenance); ensure that decisions align with organization’s goals, clinical needs, regulatory requirements, and industry best practices Training/education oversight: Ensure appropriate training/education programs are in place to support EHR users; provide resources and training materials to help users understand system capabilities, optimize workflows, and adhere to policies and procedures; establish ongoing education to help users stay up to date with system changes and updates Accountability: Assign clear roles and responsibilities to individuals or committees within organization to ensure efficient use and proper management of EHR workflows; establish mechanisms to monitor compliance and address issues or breaches that may arise Continuous improvement: Facilitate ongoing evaluation and improvement of the EHR and other related processes; establish mechanisms for gathering user feedback, monitoring system performance, identifying enhancement opportunities, and implementing necessary changes or upgrades; optimize system usability, efficiency, and effectiveness to support patient care and organization objectives Summit Discussion Questions on Governance 1  How many different governance groups should there be at an organization? The ideal structure of governance groups varies based on an organization’s size and complexity. The frequency of group meetings determines the speed at which changes can be made. Summit attendees’ organizations most commonly had six or more governance groups focused on the EHR. The number of governance groups can lead to frustration for clinical end users when they suggest EHR enhancements; what they view as a simple change may actually need to go through several committees before being implemented into the EHR. 2  How can an organization successfully involve clinical end users in the governance process (beyond submitting enhancement ideas)?  In order to successfully involve clinical end users in the governance process (beyond submitting enhancement ideas), several strategies can be employed: Have clinicians lead the governance process Ensure effective IT/informatics rounding (helps clinicians share ideas and give feedback on the EHR) Encourage end users to become advocates for ideas Have a ticket-tracking system that reports the status of tickets to end users Market the success of ideas suggested by end users (shows that their voice is heard and has led to positive EHR changes) Include end users in prioritizing ideas for IT to work on Keep lines of communication open; share status updates with end users who have dedicated time to submit enhancement ideas 3  How does your organization prioritize/determine which initiatives to push forward? In general, most healthcare organizations have a process for prioritizing EHR initiatives, but the process may not always be followed—organizations who attended the summit estimated that less than 50% of their clinicians are aware of their prioritization process. To better prioritize EHR initiatives, organizations should follow these steps: Generate ideas for enhancement that come from both leadership and end users Determine the impact the proposed change would make; if multiple departments or groups are impacted, send the idea to a governance-approval meeting (various committees may be necessary) Create a consistent scoring system to use across groups; typical criteria include (a) hours required to make a change, (b) impact on patient safety, (c) impact on clinical workflow, and (d) ability to meet regulatory requirements Clinical Informatics Summit participants determined that clinical informatics is an umbrella term that encompasses a multidisciplinary field combining healthcare, information technology, and data analysis to enhance patient care and clinician efficiency. It involves the use of technology and data management systems to collect, store, analyze, and interpret healthcare information for clinical decision-making, research, and process improvement. Clinical informatics may also be involved in workflow analysis, system design, project management, and operational readiness. Professionals in this field serve as a bridge between IT, operations, and administration areas and often act as liaisons, advisors, and advocates for clinical end users. Summit Discussion Questions on Clinical Informatics 1  What training/expertise do your clinical informaticists have? Clinical informaticists typically have a clinical background, and they also commonly have backgrounds in IT or data analytics. Many have advanced degrees in informatics. Certification with an organization’s EHR vendor is a common best practice for education. Informaticists are likely to conduct workflow analyses, work toward quality improvement initiatives, and provide EHR training and support. Most are also involved in the implementation of HIT solutions. 2  Are clinical informaticists expected to continue in their clinical role? Feedback from summit attendees indicates that it is more common for physician informaticists to maintain their clinical practice, whereas nurses more often move into a full-time informaticist role. Among summit respondents, more than half shared that their clinicians are expected to spend some of their time in the clinical setting. Superusers As defined by summit participants, an EHR superuser is an individual within a healthcare organization who possesses an extensive understanding of EHR workflows within their area of focus. Superusers are key resources for clinical end users in terms of ensuring successful adoption and optimal utilization of the EHR. Responsibilities of EHR superusers may include: Aiding communication between IT leadership and clinical end users: Disperse information to end users regarding EHR updates as well as other clinically focused IT initiatives; gather feedback from end users around common pain points and fix requests to help the IT department prioritize upcoming projects Helping users and resolving issues: Act as the first line of contact for end users who are experiencing EHR troubles; be empathetic with users, deescalate issues when necessary, build relationships, and resolve issues or route users to appropriate IT support channels Ensuring optimal workflow adoption: Help clinicians adopt optimal organization- and department-level workflows to promote EHR efficiency and standardization Assisting in system optimization/upgrades: Represent the clinician voice while testing EHR updates and provide feedback before changes go live for end users; help end users understand context around system updates Providing education/coaching: Provide subject matter expertise to trainers as they build curriculum; lead EHR education/coaching sessions as needed Supporting system configuration and customization: Have needed technical training and licenses to make changes to the EHR (changes should only impact a small number of clinicians within a given area) Note: While there are many similarities between superuser responsibilities and clinical informaticist responsibilities, the summit discussions did not delve into those similarities or the differences between these two roles. Summit Discussion Questions on Superusers 1  How does your organization maintain high superuser engagement and energy? Do you allow engagement to fluctuate with project/upgrade needs? Maintaining engagement and energy among superusers requires a combination of effective communication, proper training, incentives, flexibility, and recognition of their role’s importance. While fluctuation in engagement can be a practical approach (which many summit participants reported using), healthcare organizations should ensure superusers are equipped to handle ongoing needs and provide value beyond major projects.   2  How are superusers identified and selected at your organization? Participating organizations reported using various methods to identify and select superusers, including self-identification, leadership appointments, and peer recommendation. The goal is to ensure the superuser group contains motivated, qualified, and engaged individuals who can provide effective support to EHR users. Panel Summaries Panel 1:  Creating and Maintaining Buy-In from Senior Leadership Groups working to improve EHR user satisfaction often face challenges with needing more resources while also dealing with budget cuts. Resources are tighter than ever, and healthcare executives continually have to make difficult decisions. During the summit, Dr. Amy Maneker moderated a panel with Dr. Mark Guy, Niki Mayfield, and Dr. Ray Keller to discuss how organizations can get approval for EHR improvement projects amid budget cuts. The panel determined the following best practices for attaining leadership buy-in: Align proposal with organization goals; understand the perspective of senior leadership and look for common ground—avoid siloed approaches “There is no one tactic that works. Being aware strategically of what executives are facing in a very tumultuous time and figuring out how people can communicate effectively within their incomplete understanding of technology is important so that the executives can get a vision for how the technology is needed to do the work. For me, this is an informatics job. Informaticists have to translate technology for the executives just like they have to for clinicians and operational leaders. A lot of it comes down to communicating, understanding where folks are at, understanding the pain points, and finding the strategic moment that the executives are in.”  —Dr. Mark Guy “One of the biggest successes I have had is having an ally team. We built a team of people who are behind the project, starting from boots-on-the-ground providers and clinicians as well as an ally on the executive team who could get buy-in from that group. I particularly built a good relationship with our CIO, who was able to get me at the table of the executive team and help me understand the organization’s strategic plan. I was able to align my road map with where the organization was going. The executive team was really focusing on our ambulatory side, with every provider seeing one more patient a day. In order to get their buy-in, we focused on aligning that strategic goal with our concept of onboarding sprints and EHR training efficiency and proficiency.”  —Niki Mayfield Communicate effectively; build trust and strong relationships with all levels of the organization “It is important to be trustworthy and a person of integrity. People have to deliver previous asks and make sure they are showcasing deliverables that they have brought in. That way, when they make an appeal to executive leadership, they bring their credibility, data, strategy, and integrity. That is an important principle for me.”  —Dr. Mark Guy “Sometimes I look at decisions as a win-lose situation, but that isn’t the right way. I don’t want to be speaking to someone and looking at the situation in terms of whether I am going to win or whether I am going to lose. People should look for common ground or maybe a different path forward. If they are looking with a win-lose perspective, they will be stuck on the what, but there may be another path forward or a way to get a little win.”  —Dr. Ray Keller Use relevant data and regularly showcase success/ROI of project “It is important to build relationships and trust with your allies and team as well as communicate easy wins. As you continue to get wins, you will get more and more buy-in from the needed people. . . . Start with successes and continually communicate those successes so that people are hearing and talking about them and so that you can build on those blocks and build an ally team.”  —Niki Mayfield “One of my frequent missteps is not showcasing the good work people are doing and not bringing that progress to the leaders’ and group’s attention. In the past, I viewed that as bragging, but I think showcasing the team’s good work is really, really important. Our new leader has shown me how to do that well, with genuine sincerity.”  —Dr. Mark Guy Be persistent and passionate, but understand the importance of timing “Our network CEO was very involved in the process. We had regular meetings to talk about the project. One of the things our CEO was concerned about was our 2019 Arch Collaborative survey results, which was our first survey. The CEO was very concerned that we were rolling [Epic] out and that our providers were not happy. We knew provider satisfaction was important to our CEO. We had just completed a pilot that IT funded internally of sprint programs; we did pre- and post-measuring and had great success. The point I am making with this example is that timing is important.”  —Dr. Ray Keller “There are a lot of principles. One is timing; one is data. We have to think about what data we are using because if I am giving data to the CFO, that person is going to want data on the dollar amount.”  —Dr. Ray Keller Panel 2:  Burnout and Turnover Across the healthcare industry, burnout is a prevalent issue—which is confirmed by KLAS Arch Collaborative data as well as the lived experiences of the summit attendees and panelists. Major burnout contributors include staffing shortages, after-hours workloads, bureaucratic tasks taking up too much time, and lack of control over personal workload. Cathy McCabe moderated a panel discussing burnout and turnover with Dr. Abiodun Omoloja, Dr. Bharat Magu, Candice Larson, and Michael Tutty, PhD. The panelists shared the following best practices that their organizations are using to combat EHR-related burnout: Good “hygiene” factors (e.g., good EHR, efficient login process, team-based care) won’t prevent burnout, but poor “hygiene” can worsen burnout “American psychologist Frederick Herzberg proposed the motivator-hygiene theory of job satisfaction. He talked about motivators and hygiene factors, and the motivator factors are what make employees satisfied. . . . The hygiene factors will not make employees satisfied, but if those factors are not there, they will make employees dissatisfied. I consider the EHR to be a hygiene factor. If an organization has created a situation where the EHR works well, that does not necessarily mean clinicians will be highly satisfied. Satisfaction is driven by advancements, recognition, and purposeful work. However, if an organization has a poor EHR experience with increased after-hours work, a slow login process, and non-team-based care, these burdens can cause dissatisfaction.”  —Michael Tutty, PhD Implement a physician builder program to build autonomy and provide ability to control work “Our previous CMIO suggested that we needed a physician builder program in order to really have a positive impact. Finally, we started that program. We had one primary care physician superuser who took the lead and got trained and certified in six months. Within the first nine months, we saw dramatic improvement, especially in the large primary care area where that superuser worked. Following that, we had four other superusers who have been certified over the years. This has improved physician engagement. We now have one-third of the attrition rate of the community hospital. We have a list of physicians who are in APPs who would like to be certified physician builders.”  —Dr. Bharat Magu Assess what activities lead to burnout; appoint a chief wellness officer who can demonstrate to clinicians that the healthcare organization is actively working to reduce burnout  “We built several resources and tools at the AMA to help physicians build the business case for leadership to explain why investments in addressing burnout are a sound investment for the institution. This includes how to build the evidence, how to implement a survey to get clinician feedback, and an ROI calculator that calculates the cost of burnout for the institution. . . . We know the high cost of turnover for clinicians at all levels. For physicians, it is hundreds of thousands of dollars in recruiting costs and decreased patient access. Even when a new clinician joins, the clinician has a ramp-up time. What is harder to calculate is decreased quality and patient satisfaction. When clinicians are burned out, they are less engaged.”  —Michael Tutty, PhD “One of the things we have done is called an organizational biopsy. That has been very helpful in terms of highlighting burnout issues. That has empowered me as a CMIO to go to management and show them data that supports what I have been telling them about burnout. . . . When we send out the message, it comes from both the CMIO and our wellness officer. That tells people that we are listening and that burnout issues are important. . . . When changes occur, we try to go to the person who suggested the change and tell them their suggestion was heard. Even if we can’t fix something today, we can tell them the change is coming.”  —Dr. Abiodun Omoloja Decrease time spent in EHR by streamlining documentation and messaging processes “Pre-pandemic, clinicians spent an average of 190 minutes in the EHR during a 12-hour shift. That was a pretty significant amount of time. We were in the bottom quartile for time spent in the EHR. We had to do something, so with heightened awareness and engagement from our executive nursing leadership, so we made it a priority. Shortly after, COVID-19 happened, but we still had executive engagement and support. . . . We heard loud and clear that our nurses wanted changes and that previous change requests had gone into a black hole. We had great engagement from our frontline staff, and we would meet every two weeks to focus on prioritizing our admission history and getting that shored up. After two years, we have significantly reduced the time clinicians spend in the EMR; for a 12-hour shift, the rate is around 130 minutes per shift. Our nursing staff has more time to spend with patients or take breaks. We are pretty proud of that accomplishment.”  —Candice Larson “We have been tracking burnout in physicians since 2011, which peaked in 2014 and started going down in the subsequent years, even into the beginning of COVID-19. Lower burnout at the beginning of the pandemic was a bit of a surprise but highlighted how healthcare professionals at all levels came together to address the pandemic. But in 2021, burnout had reached its highest levels since we started measuring it a decade earlier. The healthcare sector faced staffing shortages, an increased politicization of healthcare, and the continuing stresses of delivering high-quality patient care. For example, the industry has pushed more patients to interact with the patient portal. As a result, we are seeing inbox messages increasing by more than 150% post-pandemic. This can be great for patient care but increases the number of messages for physicians to deal with on top of their busy schedule.”  —Michael Tutty, PhD Summit Attendees Healthcare Organizations Alberta Health Services Aliados Health (formerly Redwood Community Health Coalition) Allegheny Health Network AllianceChicago Aspirus Health Baptist Health Baylor Scott & White Health Breakwater Health Network Children’s Health System of Texas Children’s Healthcare of Atlanta Children’s Hospital Colorado Children’s Hospital of Philadelphia Children’s Mercy Kansas City Children’s Minnesota Children’s National Hospital Circle the City Cleveland Clinic CommonSpirit Health Community Health Care Association of New York State CommUnityCare Health Centers Confluence Health Dayton Children’s Hospital Department of Veterans Affairs ECU Health eHealth New South Wales Frances Mahon Deaconess Hospital Froedtert & the Medical College of Wisconsin Gillette Children’s Specialty Healthcare Harbin Clinic Harris Health HCA Healthcare Health Choice Network HealthLinc HonorHealth Houston Methodist Illinois Bone & Joint Institute Inova Health System INTEGRIS Health Intermountain Health Kentucky Health Center Network Lehigh Valley Health Network LifeBridge Health Loma Linda University Health Louisiana Primary Care Association Marshfield Clinic Health System Mayo Clinic Memorial Hermann Health System Mercy Health Michigan Medicine MultiCare Health System NYC Health + Hospitals OCHIN Ohio State University Wexner Medical Center Ozarks Healthcare Providence St. Joseph Health St. Jude Children’s Research Hospital SUNY Upstate Medical University Sutter Health UC San Diego Health UCSF Health UNC Health University of Illinois Hospital & Health Sciences System University of Iowa Hospitals & Clinics University of Texas MD Anderson Cancer Center University of Utah Health University of Vermont Health Network UTHealth Houston UVA Health UW Health UW Medicine Valley View Hospital Wellstar Health System Yuma Regional Medical Center Vendors and Services Firms 3M Amplifire AQuity Solutions athenahealth Chartis CHI (Investor) CSI Companies Dedalus Divurgent Epic Evergreen Healthcare Partners Goliath Technologies Greenway Health Holon Solutions InterSystems K&M Consulting MEDITECH Medix Nordic OnPoint Healthcare Partners ReMedi Health Solutions Tegria uPerform What Is the KLAS Arch Collaborative? The Arch Collaborative is a group of healthcare organizations committed to improving the EHR experience through standardized surveys and benchmarking. To date, over 300 healthcare organizations have surveyed their end users and over 420,000 clinicians have responded. Impact reports such as this one seek to synthesize the feedback from these clinicians into actionable insights that organizations can use to revolutionize patient care by unlocking the potential of the EHR.

Program Goals Build meaningful relationships between clinicians and IT Build a robust and effective governance structure Organization Outcomes 11-point increase in Net EHR Experience Score Collaborative-Verified Best Practices Clinician relationships and communication What UNC Health Did UNC Health’s Information Services Division (ISD) has built authentic partnerships with clinicians to boost overall EHR satisfaction. ISD has fostered a culture of teamwork by promoting a collective approach to inform and adapt EHR decisions, planning, and processes. ISD’s tight bonds continue to evolve, and there are numerous collaboration opportunities with clinicians, including: Meaningful engagement activities A robust shared governance model An augmented clinical informatics team Enhanced analytics resources for EHR efficiency How UNC Health Did It

Program Goals Maintain consistent satisfaction with the EHR system's response time and reliability Organization Outcomes Providers are in the 89th percentile for response time Providers are in the 90th percentile for reliability Collaborative-Verified Best Practices Reliability and response time What University of Iowa Health Care Did University of Iowa Health Care has implemented system redundancy to eliminate single points of failure in their EHR infrastructure, network, and data centers. Regular maintenance and planned replacements prevent technical debt, and updates are thoroughly tested on offline nodes before being implemented. Clinicians are informed of any service impacts, and communication templates are used to maintain consistency. Planned downtime is scheduled monthly and publicized to minimize disruption, and the organization provides consistent user environments through the use of Citrix and standardized laptops and through careful printer management. The EHR vendor is actively involved in monitoring and optimizing the platform, and upgrades are rigorously tested with the involvement of informatics staff and clinicians. This collaborative approach fosters usability and smooth transitions for clinicians. How University of Iowa Health Care Did It

What Is the Arch Collaborative Provider Guidebook? The Arch Collaborative Provider Guidebook—which focuses solely on providers (i.e., physicians, residents, fellows, and advanced practice providers)—is the result of thousands of hours of combined effort. Over 300 health systems around the world have used the Arch Collaborative survey to measure the EHR satisfaction of their providers. With the survey taking roughly 10 minutes to complete, the 145,000 providers who have participated have dedicated more than 24,000 hours to providing feedback about their EHR satisfaction. This incredible effort has allowed the Arch Collaborative to identify universal best practices that any health system—regardless of their size, region, or EHR vendor—can use to improve their providers’ EHR satisfaction. These best practices are shared here to help healthcare leaders find their organization’s own path to EHR success. While the guidebook could be seen as a checklist, it will be most effective when its principles are skillfully customized to an organization’s specific culture and circumstances. The EHR House of Success The principles in this guidebook are organized according to the Collaborative’s EHR House of Success, which includes three pillars of EHR satisfaction and a foundation. The three pillars are (1) user mastery via strong education and training, (2) an organization-wide sense of shared ownership, and (3) EHR technology that meets users’ unique needs (personalization). Collaborative analysis has shown that these three variables explain up to 70% of the variation in a clinician’s EHR satisfaction and that focusing on these key areas can greatly improve the EHR experience. Each section of this guidebook will focus on the supporting data behind a given pillar. Additionally, these pillars rest on the foundation of system reliability and response time. Dissatisfaction in these areas creates significant barriers to provider EHR satisfaction and must be addressed in order for the three pillars to effectively support provider EHR satisfaction. The guidebook also includes a section on provider burnout prevention and wellness. While the EHR is not the key driver of provider burnout, Arch Collaborative data shows that higher satisfaction with the three pillars and with the EHR correlates with higher clinician wellness and lower provider turnover. Key Changes from the 2020 Guidebook The findings in this 2023 guidebook largely support those reported in the 2020 version. Some subtle differences are noted below. The three pillars of success have evolved into the EHR House of Success.The three pillars of success now rest on the foundation of system reliability and response time—a section focused on this foundation is included in the guidebook. This guidebook is focused solely on providers. KLAS intends to share updated nursing insights in our next version of the Arch Collaborative Nursing Guidebook (click here for the 2022 guidebook). Case studies from top-performing organizations are included in each section of the Appendix. Since 2020, new questions have been added to the Arch Collaborative EHR Experience Survey and the Executive Survey (conducted with the leadership teams at healthcare organizations), and the data from those surveys is reflected in this guidebook. Some best practices (and their accompanying charts) have been removed or revised. Next Steps after Reading This Guidebook Measure with KLAS’ Arch Collaborative to determine your organization’s current EHR end-user experience Identify opportunities for improvement at your organization Use the included best practices to enhance your providers’ EHR experience Measure with KLAS’ Arch Collaborative again to see how your organization’s end-user experience has changed Data Methodology KLAS surveys clinicians using our Arch Collaborative EHR Experience Survey. This survey captures clinician feedback on various metrics, including 11 metrics (see the accompanying chart) that are aggregated into an overall Net EHR Experience Score (NEES). The NEES represents a snapshot of the clinician’s overall satisfaction with the EHR environment at their organization and can range from -100 (all negative feedback) to 100 (all positive feedback). Additionally, the insights in this guidebook draw from the following recommendations and resources: Evidence-based practices: Best practices validated by Arch Collaborative research that differentiate high-performing organizations or that have been documented to help organizations improve. Leading practices (included in Appendix): Keys to success that are commonly identified by leading organizations but have not yet been broadly validated or are too unquantifiable to be fully validated. Case studies (included in Appendix): Case studies of top-performing organizations that have worked with their vendor to improve different aspects of the EHR experience. Please note that you may find some repetition between the different sections of this guidebook. This is intentional—each section is designed to be a standalone resource for a given topic and some principles and best practices apply to more than one area. That said, improving EHR satisfaction is rarely, if ever, a single-factorial effort. To truly have an impact, healthcare organizations should implement a variety of the best practices most applicable to their organization. Creating EHR Mastery: Onboarding EHR Education Onboarding education includes training that occurs when an EHR solution is upgraded or first implemented, but it more commonly refers to the EHR training offered to newly hired providers during their first 90 days at an organization. Evidence-Based Practices Providers should receive a minimum of three hours of onboarding EHR education and would greatly benefit from eleven or more hours Provider EHR education should be taught by providers EHR education is more effective at supporting a strong EHR experience when it is workflow specific Creating EHR Mastery: Ongoing EHR Education Ongoing EHR education builds on effective onboarding education, and most principles of success for onboarding training apply to ongoing EHR training as well. Evidence-Based Practices Providers should spend 3–5 hours annually refreshing their EHR knowledge; the majority of providers do not meet this threshold The quality of the training is more important than the type of training   Providing a distraction-free learning environment leads to higher EHR satisfaction Providers want more training   Creating Shared Ownership: Provider Relationships and Communication EHR success is a journey, not a destination, so IT and informatics leaders must build strong working relationships with clinical and operational leaders. These relationships are healthiest when IT and informatics leaders create a framework within which clinical and operational leadership can successfully guide organizational goals (see Creating Shared Ownership: Governance ). There is significant overlap between the success principles for (1) provider relationships and communication and (2) shared ownership and governance. However, for readability, these topics are broken out into two sections. Evidence-Based Practices For organizations to be able to build relationships with all users, a minimum of 2–4 providers should be employed by IT/informatics (full or part time) per 1,000 provider users Consistent IT rounding works  Higher IT spending does not guarantee higher EHR satisfaction Creating Shared Ownership: Governance The success principles for shared ownership and governance are closely related to those shared above for provider relationships and communication. However, for readability, the two areas have been broken out into two sections. Evidence-Based Practices Organizations with broad, multi-disciplinary team engagement in EHR governance see higher EHR satisfaction Creating Provider Efficiency: Personalization Efficiency with the EHR is an area of acute pain for providers and one of the lowest-rated areas of EHR satisfaction. The power of personalization tools to allow organizations and individuals to meet the needs of end users without making any code changes (i.e., using functionality built into the system) is well documented in Collaborative research. We expect findings in this area to continue to expand in future years. The effect of personalization tools on standardized care and care quality has not been studied by the Collaborative. There are many personalization tools that do not impact standardized care (e.g., personalized reports and chart filters). Evidence-Based Practices Personalization tools (sometimes called “user settings”) are key to provider efficiency with the EHR   Whether they work in inpatient or ambulatory settings, providers who finish their documentation the same day report significantly higher satisfaction than those who do not Creating Provider Wellness: Reducing Burnout The mission of the Arch Collaborative is to ignite a revolution in EHR satisfaction. As part of that mission, the Collaborative measures provider satisfaction with the EHR and works to understand how the EHR experience impacts burnout. While the Collaborative is not focused on alleviating or preventing burnout, the following insights are valuable for organizations to consider as they work toward that goal. 28% of providers who have participated in the Arch Collaborative report symptoms of burnout. For providers, the EHR is a commonly reported source of burnout, after workload (including lack of control over workload and after-hours workload), time spent on bureaucratic tasks, and staffing shortages. Across clinician roles, the sense of administrative work being prioritized over clinical care is exacerbated by extra clicks, confusing screens, low-value alerts, and other EHR complexities. Organizations can lessen burnout by empowering IT teams to engage with providers to help them solve problems (whether through training or technology), ensuring providers’ voices are heard and their concerns are addressed (see also Shared Ownership: Provider Relationships and Communication and Shared Ownership: Governance ). Evidence-Based Practices Healthcare organizations should proactively optimize technology’s ability to alleviate burnout by reducing time spent on bureaucratic tasks and reducing after-hours work   Inefficient charting can increase the likelihood of burnout   On average, providers who feel their organization is doing a great job with the EHR have lower rates of burnout Providers who are completely burned out are 17 times more likely to report they are planning to leave their organization Building a Technological Foundation: System Reliability and Response Time Meeting providers’ basic technological needs—not only with the EHR but also with everything it touches (including internet and hardware)—is critical to EHR satisfaction. System reliability and response time play an important role in meeting providers’ needs, yet they don’t receive a lot of attention unless they aren’t working as intended. While most organizations report having a moderately reliable system, those with unreliable systems have significant problems. Response time is a more common pain point across Collaborative respondents and requires more focus from the organization (proactively tracking login time, using solutions from other vendors, keeping software and hardware up to date, etc.). Organizations who are most successful with their system reliability and response time seek out and manage technological recommendations from vendors and proactively address updates, fixes, and hardware replacements. Evidence-Based Practices Consistent remote access to the EHR is correlated with response-time satisfaction Poor response time can negatively impact perception of reliability, even if there is very little downtime What Is the KLAS Arch Collaborative?  The Arch Collaborative is a group of healthcare organizations committed to improving the EHR experience through standardized surveys and benchmarking. To date, almost 300 healthcare organizations have surveyed their end users and over 400,000 clinicians have responded. Reports such as this one seek to synthesize the feedback from these clinicians into actionable insights that organizations can use to revolutionize patient care by unlocking the potential of the EHR.