Jaimie Seawell, BS, DO
Internal Medicine Resident
n/a
Greensboro, North Carolina
Disclosure information not submitted.
Monica Schmidt, MPH, PhD
MT (ASCP), MPH, PhD
ConeHealth, United States
Disclosure information not submitted.
Title: Performance of the Epic In-Hospital Mortality Risk Model in the intensive care unit
Introduction: Mortality risk models are used to adjust outcomes for severity of illness differences & benchmark ICU performance. Epic Systems Corporation released their In-hospital Mortality Risk Model (IMRM) in 2018. This automated model applies a proprietary algorithm retrospectively to 49 variables obtained from EHR data. The primary aim of this study was to investigate the performance of Epic’s IMRM when applied to the ICU population at a tertiary care community hospital. The secondary aim was to see whether the automated model was able to accurately capture comorbidity and principal diagnoses data.
Methods: This is a retrospective cohort study of ICU patients treated from March 2019 through July 2021. We compared the ability of the Epic’s IMRM (Model 1) to predict mortality to the same model with full comorbidity adjustment using the Elixhauser comorbidity methodology (Model 2). The outcome was binary indicating death or discharge from ICU. We used cross-validation methodology to compare Model 1 to Model 2. We compared the Area Under the Receiver Operating Characteristic (AUROC) and Brier Score between the two models. A chart review of patients who died between Jan 1,2021 to June 30, 2021 was performed to see whether principal diagnoses and comorbidities were accurately captured by Epic’s IMRM.
Results: The primary cohort was 4,234 patient ICU stays with 58.4% Caucasian and 53.7% male. Mean age was 61.4 years. The average number of chronic conditions per patient was 5.5. The predicted mortality was 19.1% (Model 1) and 19.8% (Model 2). Actual mortality was 19.7%. There was a statistically significant difference between the AUROCs of Model 1(0.7561) and 2 (0.8149) and between the Brier scores for model 1 (0.13) and model 2 (0.12) [ t-test=p< 0.001]. Chart review of 90 ICU patients demonstrated that comorbidity was not captured by Epic’s automated IMRM 33.3% (30/90) while principal diagnosis was missed only 3.3% (3/90).