Skip to main content
bioRxiv
  • Home
  • About
  • Submit
  • ALERTS / RSS
Advanced Search
New Results

Characterizing Inpatient Medicine Resident Electronic Health Record Usage Patterns Using Event Log Data

View ORCID ProfileJonathan H. Chen, Jason K. Wang, David Ouyang, Jason Hom, Jeffrey Chi
doi: https://doi.org/10.1101/428169
Jonathan H. Chen
1Assistant Professor, Department of Medicine (Biomedical Informatics), Stanford University
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Jonathan H. Chen
Jason K. Wang
2Undergraduate, Mathematical and Computational Science Program, Stanford University
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
David Ouyang
3Cardiology Fellow, Department of Medicine, Stanford University
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Jason Hom
4Clinical Assistant Professor, Department of Medicine, Stanford University
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Jeffrey Chi
5Clinical Associate Professor, Department of Medicine, Stanford University
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • Abstract
  • Full Text
  • Info/History
  • Metrics
  • Preview PDF
Loading

ABSTRACT

Amid growing rates of burnout, physicians report increasing electronic health record (EHR) usage alongside decreasing clinical facetime with patients. There exists a pressing need to improve physician-computer-patient interactions by streamlining EHR workflow.

To identify interventions to improve EHR design and usage, we systematically characterize EHR activity among internal medicine residents at a tertiary academic hospital across various inpatient rotations and roles from June 2013 to November 2016.

Logged EHR timestamps were extracted from Stanford Hospital’s EHR system (Epic) and cross-referenced against resident rotation schedules. We tracked the quantity of EHR logs across 24-hour cycles to reveal daily usage patterns. In addition, we decomposed daily EHR time into time spent on specific EHR actions (e.g. chart review, note entry and review, results review).

In examining 24-hour usage cycles from general medicine day and night team rotations, we identified a prominent trend in which night team activity promptly ceased at the shift’s end, while day team activity tended to linger post-shift. Across all rotations and roles, residents spent on average 5.38 hours (standard deviation=2.07) using the EHR. PGY1 (post-graduate year one) interns and PGY2+ residents spent on average 2.4 and 4.1 times the number of EHR hours on information review (chart, note, and results review) as information entry (note and order entry).

Analysis of EHR event log data can enable medical educators and programs to develop more targeted interventions to improve physician-computer-patient interactions, centered on specific EHR actions.

INTRODUCTION

As medical educators, we hope our resident trainees value direct patient care and contact. Instead, we may progressively find their attention dominated by electronic health records (EHR) that mediate their work and proxy for their patients.1 Observational studies confirm an increasing shift from direct patient care to computer use in the wake of duty hour restrictions,2,3,4 informing ongoing reforms into the structure of medical training.5 Physicians report increasing time spent on paperwork and the computer6, alongside less time available for clinical interactions with patients.7 A growing burden of redundant clinical notes, alert fatigue, and an overflowing inbox has led to a systemic “4000 clicks a day” problem8 that has contributed to physician job dissatisfaction and burnout rates. Indeed, correlations between an increasing EHR task load and physician burnout have been demonstrated.6,9 There exists a pressing need to understand clinical EHR usage to inform opportunities to improve effective patient care processes.

Previous studies have quantified EHR usage intensity through direct observation and self-reported diaries.2,10 Ironically, with the amount of time trainees spend on the EHR, the computer also provides precise, reproducible, and scalable quantification of their electronic activities. Prior comparisons between direct observation of clinician activity and logged EHR timestamps confirm that the two approaches yield similar results for workflow analysis.11,12 Here, we use timestamp data to systematically characterize the intensity of EHR usage across different inpatient medicine rotations and roles.

METHODS

Logged timestamps from the Epic EHR system for inpatient rotations of internal medicine residents at an academic tertiary care hospital were extracted from the STRIDE project13 from June 2013 to November 2016. We tracked the quantity of logged EHR actions accessed over 24-hour cycles per user binning by half-hour intervals. Logged EHR actions correspond to behaviors performed on the EHR as clinicians navigate components (e.g. notes, orders, results) of a patient’s electronic chart. To estimate time spent per action, we considered time intervals between access logs separated by 5 minutes or less of inactivity. Sensitivity of results relative to time intervals up to 10 minutes was also considered. We binned EHR actions into common behavioral categories and computed mean time spent per category per work day. Daily EHR usage was estimated as the sum of all active inter-access time intervals per day. User timestamps were cross-referenced against resident year and rotation schedules to account for each user’s progression through the internal medicine residency program over the three-year window. User-days with less than one hour of activity were excluded from analysis to account for remote access during vacation days. P-values were computed using two-sample t-tests allowing unequal variances (Welch’s t-test). Analyses were performed with Python 2.7 and R 2.13. The study was deemed non-human subjects research by the Stanford Institutional Review Board.

RESULTS

During the three-year window, 15,909,629 unique actions were logged by 101 unique residents covering 99 PGY1 (post-graduate year one) intern-years and 61 PGY2+ resident-years. Fig 1 illustrates the intensity of EHR interactions per resident day during a 24-hour cycle for PGY1 interns versus PGY2+ residents across different inpatient rotations.

Fig 1:
  • Download figure
  • Open in new tab
Fig 1:

EHR Actions Vs. Time of Day

Table 1 highlights mean daily EHR usage times, decomposed by mean time spent on common EHR action categories across each inpatient rotation and role. Across all rotations, trainees spent a mean of 5.38 hours (PGY1=5.47, PGY2+=5.21) with standard deviation=2.07 (PGY1=2.10, PGY2+=2.02) and median of 5.33 hours (PGY1=5.47, PGY2+=5.09) with interquartile range=3.85-6.85 (PGY1=3.90-6.97, PGY2+=3.80-6.65) per work day using the EHR. Medical chart reviews account for the largest portion of activity (>40% on average) followed primarily by note entry and review. PGY1 interns spent disproportionately more time on note entry compared to PGY2+ residents during general medicine (15.2% vs. 4.0% of daily EHR usage, P<0.001) and emergency medicine rotations (18.4% vs. 13.9%, P<0.001), but similar times during the night team rotation (11.9% vs. 11.3%, P=0.28). Conversely, across all three rotations, PGY2+ residents spent disproportionately more time on note review compared to PGY1 interns (general medicine=20.7% vs. 13.0%, P<0.001; night team=21.8% vs. 11.6%, P<0.001; emergency medicine=11.6% vs. 8.1%, P<0.001). Trainees spent roughly 7.6 minutes (PGY2+ only), 9.7 minutes (PGY1=12.3, PGY2+=6.6), 3.7 minutes (PGY1=3.5, PGY2+=4.2), and 7.2 minutes (PGY1=8.0, PGY2+=5.5) on chart review actions per patient across intensive care unit, general medicine, night team, and emergency department rotations, respectively.

View this table:
  • View inline
  • View popup
Table 1: EHR Usage Intensity Across Inpatient Medical Rotations and Roles Per User-Daya

EHR usage summary statistics and mean time spent on common EHR action categories per user-day across different inpatient rotations for PGY1 interns and PGY2+ residents. Time spent on category-specific actions are also shown as a percentage of mean daily EHR usage time. The EHR Navigator consists of pre-curated sequences of modules to facilitate common actions such as admission, rounding, and discharge.

Daily EHR usage estimations prove robust to the choice of inactivity cutoff; the standard deviation of mean daily EHR usages computed using inactivity cutoffs from 5-10 minutes is 28.8 minutes. The choice of a 5-minute cutoff captures the vast majority of active computer sessions (98.4% of inter-access time intervals), while excluding lengthy idle sessions. Excluding user-days based on an activity threshold of one hour removes 11.6% of all user-days (435/3756), corresponding to roughly one day off per week.

DISCUSSION AND CONCLUSION

With increasing reliance on EHRs to mediate patient care, direct analysis of EHR audit logs provides a granular and objective way to characterize physician EHR usage. A key limitation of this approach is the estimation of idle time between access logs; overestimating idle times could overestimate EHR usage times. However, our sensitivity analysis showed that estimations were robust to cutoffs between 5-10 minutes. Additionally, although results are aggregated across multiple years, the reported EHR usage statistics are derived from a single institution.

The pattern of EHR activity over 24-hour cycles provides qualitative insights into resident behavior (Figure 1). The 2011 ACGME duty hour restrictions prompted the separation of the hospital’s General Medicine rotation into Day and Night Teams. These 24-hour cycles may suggest that night teams treat patient care as discrete shift work, with clinical activities promptly ceasing at 7 AM as noted by the steep drop-off, while general medicine (day team) EHR activity often lingers well beyond duty hour recommendations (9 PM onwards) and restrictions (11 PM onwards), defined as ten and eight hours before the subsequent 7 AM shift.

Perhaps more concerning is that the prevalence of physician-computer interaction almost certainly restricts direct physician-patient interaction. Assuming a 12-hour work day3, PGY1 interns and PGY2+ residents spent nearly half (PGY1=46%, PGY2+=43%) of work time on the EHR, respectively. This is especially notable with peaked EHR activity when Day Teams arrive, suggesting the traditional model of pre-rounding at the patient bedside has been replaced by the workroom computer as the trusted source for patient information.1

Comparing between roles (Table 1), we see that PGY2+ residents execute a greater number of EHR actions with disproportionately more time spent on note review, yet less daily EHR time, compared to PGY1 interns. Indeed, in the clinic, PGY2+ residents embrace a supervisory role and often oversee twice the number of patients (note the nearly 2:1 median patient ratio during general medicine rotations) compared to PGY1 interns. Conversely, PGY1 interns are delegated time-intensive note entry duties albeit for a smaller population of patients, potentially accounting for greater daily EHR time. Across diverse rotations, we see significant time spent on the same pattern of EHR activities. PGY1 interns and PGY2+ residents spent on average 2.4 and 4.1 times the number of EHR hours on information review (chart, note, and results review) as information entry (note and order entry) (Table 1), respectively. Improvements in EHR design could target the burden of data retrieval by encouraging more concise documentation and redesigned or automated content organization.14 By identifying specific EHR activities that consistently dominate resident computer usage across multiple inpatient rotations and roles, we hope to facilitate a more targeted, data-driven approach to improving physician-computer-patient interactions.

Contributions Statement

JHC conceived the study and design. JHC and JKW performed analyses and drafted the initial manuscript. All authors reviewed the manuscript and contributed to study design.

ACKNOWLEDGEMENTS

Project supported by the Stanford Cyber Initiative. J.H.C was supported in part by the NIH Big Data 2 Knowledge Award Number K01ES026837 through the National Institute of Environmental Health Sciences.

Patient data extracted and de-identified by Gomathi Krishnan of the STRIDE (Stanford Translational Research Integrated Database Environment) project, a research and development project at Stanford University to create a standards-based informatics platform supporting clinical and translational research. The STRIDE project described was supported by the National Center for Research Resources and the National Center for Advancing Translational Sciences, National Institutes of Health, through grant UL1 RR025744.

Content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH, or Stanford Healthcare.

REFERENCES

  1. 1.↵
    Chi J. & Verghese, A. Clinical education and the electronic health record: the flipped patient. JAMA 312, 2331–2 (2014).
    OpenUrlCrossRefPubMed
  2. 2.↵
    Block L. et al. In the wake of the 2003 and 2011 duty hours regulations, how do internal medicine interns spend their time? J. Gen. Intern. Med. 28, 1042–1047 (2013).
    OpenUrlCrossRefPubMed
  3. 3.↵
    Ouyang D., Chen J. H., Hom J., & Chi J. Internal medicine resident computer usage: an electronic audit of an inpatient service. JAMA Internal Medicine. 176, 252–54 (2016).
    OpenUrl
  4. 4.↵
    Desai, S. V., Asch D. A., Bellini L. M., et al. Education Outcomes in a Duty-Hour Flexibility Trial in Internal Medicine. The New England Journal of Medicine. 10, 1056 (2018).
    OpenUrl
  5. 5.↵
    Asch D. A., Bilimoria K. Y. & Desai S. V. Resident Duty Hours and Medical Education Policy — Raising the Evidence Bar. N. Engl. J. Med. 376, 1704–1706 (2017).
    OpenUrl
  6. 6.↵
    Babbott S., Manwell L.B., Brown R., et al., Electronic medical records and physician stress in primary care: results from the MEMO Study. J Am Med Inform Assoc. 21, 1006 (2014).
    OpenUrl
  7. 7.↵
    Friedberg M., Chen P.G., Van Busum K.R., et al. Factors Affecting Physician Professional Satisfaction and Their Implications for Patient Care, Health Systems and Health Policy. RAND Corporation, 2013.
  8. 8.↵
    Hill R.G., Jr., Sears L.M., & Melanson S.W. 4000 clicks: a productivity analysis of electronic medical records in a community hospital ED. Am JEmergMed. 31, 1591–1594 (2014).
    OpenUrl
  9. 9.↵
    Shanafelt T.D., Dyrbye L.N., Sinsky C., et al. Relationship between clerical burden and characteristics of the electronic environment with physician burnout and professional satisfaction. Mayo Clin Proc. 91, 836–848 (2016).
    OpenUrlCrossRefPubMed
  10. 10.↵
    Sinsky, C., Colligan L., Li L., et al. Allocation of Physician Time in Ambulatory Practice: A Time and Motion Study in 4 Specialties. Annals of Internal Medicine 165, 753–60 (2016).
    OpenUrlPubMed
  11. 11.↵
    Hribar, M. R., Read-Brown S., Goldstein I. H., et al. Secondary Use of Electronic Health Record Data for Clinical Workflow Analysis. Journal of the American Medical Informatics Association. 25, 40–46 (2018).
    OpenUrl
  12. 12.↵
    Arndt, B.G, Beasley, J.W., Watkinson M.D., et al. Tethered to the EHR: Primary Care Physician Workload Assessment Using EHR Event Log Data and Time-Motion Observations. Annals of Family Medicine. 15, 419–26 (2017).
    OpenUrlAbstract/FREE Full Text
  13. 13.↵
    Lowe, H. J., Ferris, T. a, Hernandez, P. M. & Weber, S. C. STRIDE--An integrated standards-based translational research informatics platform. AMIA Annu. Symp. Proc. 2009, 391–5 (2009).
    OpenUrlPubMed
  14. 14.↵
    Hirsch, Jamie S., Jessica S. Tanenbaum, Sharon Lipsky Gorman, Connie Liu, Eric Schmitz, Dritan Hashorva, Artem Ervits, David Vawdrey, Marc Sturm, and Noémie Elhadad. 2015. “HARVEST, a Longitudinal Patient Record Summarizer.” Journal of the American Medical Informatics Association: JAMIA 22 (2): 263–74.
    OpenUrlCrossRefPubMed
Back to top
PreviousNext
Posted September 26, 2018.
Download PDF
Email

Thank you for your interest in spreading the word about bioRxiv.

NOTE: Your email address is requested solely to identify you as the sender of this article.

Enter multiple addresses on separate lines or separate them with commas.
Characterizing Inpatient Medicine Resident Electronic Health Record Usage Patterns Using Event Log Data
(Your Name) has forwarded a page to you from bioRxiv
(Your Name) thought you would like to see this page from the bioRxiv website.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Share
Characterizing Inpatient Medicine Resident Electronic Health Record Usage Patterns Using Event Log Data
Jonathan H. Chen, Jason K. Wang, David Ouyang, Jason Hom, Jeffrey Chi
bioRxiv 428169; doi: https://doi.org/10.1101/428169
Reddit logo Twitter logo Facebook logo LinkedIn logo Mendeley logo
Citation Tools
Characterizing Inpatient Medicine Resident Electronic Health Record Usage Patterns Using Event Log Data
Jonathan H. Chen, Jason K. Wang, David Ouyang, Jason Hom, Jeffrey Chi
bioRxiv 428169; doi: https://doi.org/10.1101/428169

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
  • Tweet Widget
  • Facebook Like
  • Google Plus One

Subject Area

  • Epidemiology
Subject Areas
All Articles
  • Animal Behavior and Cognition (4235)
  • Biochemistry (9136)
  • Bioengineering (6784)
  • Bioinformatics (24001)
  • Biophysics (12129)
  • Cancer Biology (9534)
  • Cell Biology (13778)
  • Clinical Trials (138)
  • Developmental Biology (7636)
  • Ecology (11702)
  • Epidemiology (2066)
  • Evolutionary Biology (15513)
  • Genetics (10644)
  • Genomics (14326)
  • Immunology (9483)
  • Microbiology (22839)
  • Molecular Biology (9090)
  • Neuroscience (48995)
  • Paleontology (355)
  • Pathology (1482)
  • Pharmacology and Toxicology (2570)
  • Physiology (3846)
  • Plant Biology (8331)
  • Scientific Communication and Education (1471)
  • Synthetic Biology (2296)
  • Systems Biology (6192)
  • Zoology (1301)