Jamie Palumbo, MA
Data Scientist
VPS, LLC
Los Angeles, California
Disclosure information not submitted.
Gerardo Soto-Campos, MS, PhD
Director of Analytics
Virtual Pediatric Systems, LLC, United States
Disclosure information not submitted.
Tom Rice, BS, MD
Professor, Pediatrics (Emeritus)
Medical College of Wisconsin, United States
Disclosure information not submitted.
Randall Wetzel, BS, MB, MBA, FCCM
Professor of Anesthesiology and Pediatrics
Children's Hospital of Los Angeles
Los Angeles, California, United States
Disclosure information not submitted.
Title: Determining Risk Factors and Predicting Extubation Failure Using Historical Data
Introduction: Extubation failure rates in U.S. PICUs vary from 5-9%. A tool to assess the risk of extubation failure could reduce the risk of adverse effects from reintubation. Furthermore, a model predicting successful extubation would help specify a standardized risk for extubation failure in benchmarking studies. This work presents a method to discover extubation failure covariates and builds two models to predict extubation failure.
Methods: 98,749 PICU cases with an endotracheal tube (ETT) from 144 units spanning 2017 to 2020 were obtained from the Virtual Pediatric Systems registry. Cases were excluded if a tracheostomy, death prior to extubation, or end of life order was recorded, reducing the cohort to 88,037. The outcome extubation failure was defined as the planned removal and subsequent reinsertion of an ETT within 24 hours (Kurachek, 2003). Data was divided into independent training and testing sets with a 75/25 split. The training set was used to simulate 100 bootstrap forward stepwise logistic regressions with both Akaike and Bayesian Information Criteria (AIC/BIC). Potential covariates included PIM3 and PRISM III admission variables and those proposed by Kurachek. The variance inflation factor controlled for collinearity; only significant variables (p< 0.05) were retained. The final covariates were the set of variables surviving the bootstrap filters with a frequency of at least 30%. These covariates were used to generate the final coefficients using the training set. Discrimination and calibration of the models were assessed with AUROC, Hosmer-Lemeshow, and GiVITI calibration belt using the testing set. All simulations were run in R on an AWS virtual machine.
Results: The bootstrap chose a subset of PIM3, PRISM III, and Kurachek variables. Each bootstrap simulation took nearly 24 hours to run. Both the AIC and BIC models had AUROC of 0.81, and a standardized ratio of predicted to actual extubation failure of 0.99 in the testing set. The Hosmer-Lemeshow p-values were 0.06 and 0.16, respectively. GIVITI showed excellent calibration.
Conclusions: The discrimination, calibration, and standardized ratio of our predictive models are promising. While computationally demanding, our method for choosing extubation failure covariates provides a potential approach to complement physicians’ clinical expertise.