When the Statistics Steal the Show

We read the publication by Farag et al.¹  entitled “Comparison of Three Techniques for Ultrasound-guided Femoral Nerve Catheter Insertion: A Randomized, Blinded Trial” with interest and feel compelled to share our views with the readership of Anesthesiology.

The intention of this letter is not to criticize the study, but to reflect on an increasingly common problem where a complex statistical analysis obscures the clinical relevance of a simple question. This large study was conducted like a National Institutes of Health clinical trial using the analytic capabilities of Ph.D.-level statisticians. We appreciated the statistical treatment of measurements taken over time. For instance, the verbal response scale was averaged for each patient using a time-weighted formula, and opioid administrations were converted to morphine equivalents and totaled. This statistical approach ideally would be followed even in studies with smaller sample sizes. We also appreciated the authors’ recognition that not all variables are normally distributed and that they took appropriate steps to account for this in their statistical analysis. Specifically, the distribution of total opioid requirement was found to be lognormal; hence, the effect of catheter insertion technique estimates “the ratio (or percent difference) of geometric means.”¹  In addition, opioid requirement was tested in “a linear regression model of log-transformed total intravenous morphine equivalent observations as the response and randomized catheter insertion technique as the treatment of interest.”¹ 

Nonetheless, although the study aimed to answer a simple clinical question, the article is so heavily statistical that clinicians are likely to gloss over much of it in search of what is relevant to their practice. For instance, the juxtaposition of “noninferiority” and “superiority” throughout the article is a challenge to read and keep track of, particularly for those without a strong background in statistics. Although crucial to the statistical approach, we believe that many clinicians may gloss over these terms without truly understanding their meaning. Moreover, although the authors appropriately document and cite the newer (Mascha & Turan) and older (Holm, O’Brien) statistical procedures that are less likely to be familiar to the readership, we remain doubtful that these would be meaningful to the average clinician who may be interested in applying these findings to their clinical practice.

Despite the complex and comprehensive statistical analysis, we feel that the study design does not actually allow for any clinically meaningful conclusions on the choice of catheter. This is because most patients having total knee arthroplasty with femoral nerve block alone have residual pain in the sciatic territory² ; something the authors did not control for in a systematic way.¹  Because nearly all patients will have pain, the methodology is inherently biased toward finding no difference between groups. Moreover, there is no information about the success or duration of the blocks as this was not tested at all—without information on sensory or motor distribution, it is not possible to draw conclusions on the success of the technique.

Most fundamentally, the research questions asked are of little relevance for today’s anesthesiologist. The study was conducted over several years during which time substantial changes in techniques and technology have occurred. The question of whether or not to stimulate via the catheter was relevant before the advent of ultrasound guidance in regional anesthesia. However, with ultrasound, the primary determinant of whether a catheter is adequately positioned is not the presence or absence of evoked motor response but visualization of anesthetic spread in the desired tissue plane.^3,4  In addition, it seems obvious that techniques that require extra steps and more equipment will take longer to perform and have additional costs. So, testing hypotheses of increased block performance time and costs focuses on significance or nonsignificance based on the P value and takes us away from appreciating that a difference in mean block performance time of approximately 1 min may not have a whole lot of clinical relevance.

Finally, the design and conclusions seem “to close the door” to further novel research—for example, with respect to different techniques (in-plane vs. out-of-plane), equipment (different needle gauges and catheter designs), volume of injectate, and research outcomes such as safety. Indeed, by advocating the use of ultrasound alone for femoral catheter placement, the authors ignore the potential safety benefits of using nerve stimulation to warn the clinician of potentially harmful needle-nerve contact and intraneural injection.⁵  In summary, this article demonstrates that even when meticulous data collection and advanced statistical analysis are applied to a research design with little clinical relevance, the practicing clinician may be left without any data of significance to their clinical practice.

The authors declare no competing interests.