Conor Morin
Candidate for Doctor of Pharmacy 2022
MCPHS University
Boston, Massachusetts
Disclosure information not submitted.
Anirudh Padki, n/a
Candidate for Doctor of Pharmacy 2022
MCPHS University, United States
Disclosure information not submitted.
Todd Miano, PharmD, PhD
Critical Care Pharmacist
University of Pennsylvania Perelman School of Medicine
Philadelphia, Pennsylvania, United States
Disclosure information not submitted.
Sandra Kane-Gill, MSc, PharmD, FCCP
Professor of Pharmacy and Therapeutics
University of Pittsburgh School of Pharmacy
Pittsburgh, Pennsylvania, United States
Disclosure information not submitted.
Robert Deveau, PharmD
Clinical Pharmacist
BIDMC-Boston
Boston, Massachusetts, United States
Disclosure information not submitted.
Gabrielle Cozzi, PharmD, BCCCP
Clinical Pharmacist
BIDMC-Boston, United States
Disclosure information not submitted.
.jpg "Adrian Wong, BCCCP, MPH, PharmD photo")
Adrian Wong, BCCCP, MPH, PharmD
Assistant Professor
MCPHS University
Boston, Massachusetts
Disclosure information not submitted.
Title: Comparison of COVID-19 Pre-print and Published Versions of Studies on Therapies for ICU Patients
Introduction: Pre-print articles are open-access versions of manuscripts available prior to peer-review. There has been a significant recent increase in the amount of pre-print articles, given the need for rapid dissemination of data needed to treat COVID-19. The purpose of this study was to compare pre-print and published articles to evaluate for similarities and differences to determine the utility of pre-print articles to inform clinical practice.
Methods: Using Altmetric Explorer, articles were identified that focused on the three NIH COVID-19 guideline recommended therapies (dexamethasone, remdesivir, tocilizumab). Two independent reviewers evaluated the articles for the following criteria: 1. Available in English, 2. Included critically ill patients, 3. Were observational or randomized trials, and 4. Pre-print available. A random selection of up to 5 articles per therapy were included for further evaluation. The primary evaluation metric was the incidence of change in a study’s reported primary outcome (statistical significance, effect measure). Secondary evaluation metrics were time from pre-print to publication, percent similarity between pre-print and published versions (using Turnitin), incidence of change in reported secondary outcomes (statistical significance, numerical value), and change in study conclusions. A 95% CI was estimated for investigated evaluation metrics.
Results: 774 Altmetric items were screened, with 28 (4%) meeting inclusion criteria, and 14 randomly selected for evaluation. The median Altmetric score was 13 (IQR 2-218.5). No articles had a change in statistical significance of the primary outcome (0%, 95% CI 0-0.23), although 4 (29%, 95% CI 0.08-0.58) had a change in the effect measure. There was a median of 110 days (IQR 60.5-124) from pre-print to publication. The median percent similarity between the pre-print and published versions was 64.5% (IQR 51.5-69.3). Four studies (29%, 95% CI 0.08-0.58) had a change in statistical significance for at least one secondary outcome. Two studies had a change in their conclusion (14%, 95% CI 0.02-0.43).
Conclusions: These findings suggest that pre-print, compared to published versions of articles, do not have a dramatic change in their primary outcomes. However, interpretation of secondary outcomes should be done with caution.