When Large Administrative Databases Provide Less Relevant Information than Randomized Studies

We read with interest the retrospective study conducted by Wasserman et al.,¹  based on a national administrative database assessing the impact of intravenous acetaminophen on perioperative opioid utilization and outcomes in patients undergoing open colectomies.

Research based on administrative data sets can provide information of major importance for clinical practice, but the interpretation of results is difficult, and causal inference is circumscribed by intrinsic methodologic limitations. In this study,¹  we observed three main limitations with potential impact on result interpretation: (1) the validity of main outcome data (morphine consumption) is questionable compared with monitored clinical studies, (2) the doses of acetaminophen administered in the treated group were heterogeneous, and (3) the estimation of treatment effect is likely to be biased by uncontrolled confounding factors. The sensitivity analysis provided by the authors is not enough to provide an unbiased estimation of treatment effect. To minimize bias, a propensity score analysis²  or another sophisticated multivariable matching process³  should have been performed, because the patients who received acetaminophen differed markedly from those who did not. Despite the large sample size (n = 181,640),¹  we believe that the average treatment effect estimation is not robust enough to support any practice recommendations based on this study.  Therefore, the amount of new information is relatively limited.

Although we thank the authors for not stating recommendations based on their results, we respectfully disagree with their conclusions: “Important next steps include validation of these results with alternative data and identifying patients and administration schedules (e.g., routine IV acetaminophen every 6 h, dosing for 48 h) most likely to result in benefit.”¹  The largest randomized control trial (n = 550 patients) evaluating the treatment effect of a homogeneous and appropriate dose of acetaminophen demonstrated a reduction in morphine requirements greater than the threshold prespecified by Wasserman et al.¹  (−31%; P < 0.001) and was not cited.⁴  Citing appropriate references allows readers to understand new results and interpret them while taking into account results obtained using a high level of evidence-based studies. Such an approach in the reporting research would limit the frequent claims for more and bigger studies, when results are already available in the literature.

Large administrative databases provide a vast amount of data reflecting our clinical practice. However, only powered, randomized, controlled trials provide unbiased estimation of treatment effect. In this clinical setting (i.e., impact of intravenous acetaminophen on postoperative requirements), we do not believe that additional observational cohort analyses or additional randomized studies or meta-analyses⁵  are a priority, because we already have the response.⁴ 

Last, although Wasserman et al.¹  also studied the incidence of outcome (i.e., opioid-related adverse effect) and not only morphine consumption, we still consider that opioid consumption is definitely not a clinically relevant primary endpoint and could not be an intermediate outcome of a patient-related optimal one.⁴  Taking into account the lack of demonstrated effect of acetaminophen on opioid-related adverse effect, this drug probably has minimal clinically relevant effects in the early perioperative period.

The authors declare no competing interests.