THE current issue of Anesthesiology contains a report of a laboratory study 1that addresses the question, “Is it appropriate to treat neuropathic pain with spinal opioids?” This editorial examines the data that address this question, and the possibility that the differences between laboratory studies reporting positive and negative effects may be attributed, at least in part, to experimenter bias. We recognize that efficacy with opioids, and perhaps other analgesics, may be lost over time with chronic treatment, and that the results of opioids may also differ according to patient population, laboratory model, and route of administration. Therefore, we focus this discussion on whether spinal delivery of opioids has analgesic effects, regardless of the duration of action, in the commonly used spinal nerve ligation model of neuropathic pain. 2
Initial clinical reports of chronic intrathecal morphine treatment, published 20 yr ago, showed both acute and sustained efficacy in patients with chronic pain, including neuropathic pain, although dose escalation occurred with time. 3,4At the same time, others reported little or no analgesic effects of spinal morphine in neuropathic pain patients. 5Later reports suggested that spinal opioids produce partial pain relief in neuropathic pain patients, 6,7and other studies reported that intrathecal opioids produce substantial or good analgesic effects in patients with neuropathic pain, even after long-term administration. 8,9Thus, the clinical literature regarding efficacy of spinal opioids for neuropathic pain has evolved from initial good efficacy, to poor efficacy, and, more recently, back to good efficacy in many patients.
A laboratory model of neuropathic pain in rats in which ligation of low lumbar spinal nerves results in reduced withdrawal threshold to tactile stimulation of the paw, akin to mechanical allodynia in patients with neuropathic pain, was first described over a decade ago. 2Initial studies using this model showed it to be sympathetic-dependent, with hypersensitivity resolving after chemical or surgical sympathectomy. 10Several laboratories, including ours, showed that the spinal nerve ligation model was also resistant to treatment with intrathecal morphine. 11–13These were considered important validation studies because neuropathic pain was at that time thought to frequently be sympathetic-dependent and, as noted above, resistant to treatment with intrathecal morphine.
This Editorial View accompanies the following article: Zhao C, Tall JM, Meyer RA, Raja SN: Antiallodynic effects of systemic and intrathecal morphine in the spared nerve injury model of neuropathic pain in rats. Anesthesiology 2004; 100:905–11.
Neuropathic pain is now considered to be only infrequently sympathetic-dependent and often responds to spinal opioids, and these changes in understanding of clinical neuropathic pain have been mirrored by changes in results obtained in this spinal nerve ligation model. For example, the current authors more recently noted that the effect of sympathectomy was smaller than originally described and more variable, depending on rat strain, 14and we 15and others 16failed to observe an effect of sympathectomy on hypersensitivity using this model. In addition, the current report 1demonstrates full efficacy of intrathecal morphine to reduce hypersensitivity to mechanical stimulation in rats with spinal nerve ligation or spared nerve injury, another model of neuropathic pain. The dose of intrathecal morphine found to be effective was small, and, as noted by the authors, was similar to that needed to treat nonneuropathic acute and chronic pain in these species.
It is somewhat reassuring that the data from the spinal nerve ligation model now seem to be consistent with the growing consensus that neuropathic pain is not very sympathetic-dependent and that spinal opioids are often effective in patients with neuropathic pain. However, these findings also raise the concern that the results obtained in studies with the spinal nerve ligation model of neuropathic pain may be affected by the expectations of the experimenters at the time the studies are conducted. In fact, Zhao et al. 1were careful to use blinding procedures to prevent experimenter bias from affecting their results, and they suggest that experimenter bias could account for the negative results of earlier studies. Results of studies conducted in our laboratory support this suggestion. We previously reported that intrathecal morphine was ineffective after spinal nerve ligation, 12but in a recent, rigorously blinded replication of our initial nonblinded study, we detected analgesic effects of intrathecal opioids (unpublished data) at doses similar to those reported by Zhao et al. 1
In addition to the differences in blinding procedures, many other factors could also contribute to the differences between the results of these studies. For example, even seasonal changes in the source of the protein included in commercial rodent chow—with no change in the total protein or calorie content—can significantly affect the results of studies with these laboratory pain models. 17It is possible that such subtle differences in diet or some other unidentified and uncontrolled factor could also contribute to the differences in the results between these studies, so it is not appropriate to conclude that all of the differences in the results of the studies with the spinal nerve ligation model discussed above can be attributed to experimenter bias in the nonblinded studies.
However, experimenter bias is a real phenomenon that has been clearly demonstrated in clinical research. For example, in a clinical trial for multiple sclerosis, patient evaluations performed by nonblinded clinicians detected statistically significant therapeutic effects, but patient evaluations performed in the same study by blinded clinicians revealed that the treatment did not produce beneficial effects. 18Apparent therapeutic effects are often observed in initial, small, nonblinded clinical trials, but not in larger, blinded trials. For example, in a series of clinical studies conducted to test the therapeutic potential of a monoclonal antibody for rheumatoid arthritis, all of the initial, small, nonblinded trials reported therapeutic effects of the antibody. 8Subsequently, three large, blinded experiments, some of them conducted by the same investigators who had conducted some of the initial nonblinded studies, all failed to detect any beneficial effect of the treatment. 19
This scenario is so common in clinical trials that it is not considered at all surprising. In these cases, the beneficial results of the treatment reported in the nonblinded trials are typically attributed to experimenter bias. For this reason, decisions about whether clinical treatments are truly effective are often based only on the most well controlled studies. For example, systemic opioids are now considered one of the first-line treatments for neuropathic pain, but that decision was based on only five recent studies, all of which were large, randomized, double-blind, placebo-controlled clinical trials (for review see Dworkin et al. 20). Clearly, there is a need for strict controls to prevent experimenter bias, which we recognize when testing the potential efficacy of treatments in humans.
Classic studies published more than 40 yr ago demonstrated that experimenter bias could also affect the results of laboratory studies. For example, in a conditioning study, planaria were given a 3-s light cue followed by a 1-s shock. Observers recorded whether the planaria made contractions or head turns during the light cue, which were the dependent measures used to determine if the planaria were learning to associate the light with the shock. Increasing numbers of contractions and head turns in response to the light would be evidence of classic conditioning. Observers who had been told to expect rapid conditioning recorded significantly increased numbers of anticipatory movements, compared with observers who had been told to expect little or no evidence of conditioning. 21In a learning study in rats, experimenters trained rats to run to the dark arm of a T-maze for a food reward. Rats were given 10 trials each day for 5 days. Experimenters who were told to expect their rats to demonstrate the task quickly did in fact record faster acquisition and better performance in their rats than experimenters who had been told to expect that their rats would be poor learners. 22In another learning study, rats were trained to perform a series of tasks for a food reward in an operant or “Skinner” box. If the experimenter had been told that the rats had been bred for good performance in these tasks, better performance was recorded for the rats than if the experimenter had been told that the rats had been bred for poor performance in these tasks. 23Anyone who has conducted tail flick or formalin testing in rodents recognizes that the expectations of the investigator can affect the results of laboratory pain tests as well. In the case of paw withdrawal in the relatively unrestrained animal, which often raises the paw to walk, groom itself, or make postural adjustments (e.g. , in the Hargreave or von Frey test), the influence of investigator bias can be even greater.
Experimenter bias is not limited to the time when the behavioral observations are recorded; it can also affect nonbehavioral measurements. For example, standard laboratory procedures used to count blood cells were shown to require a degree of consistency that was not possible, given the equipment and procedures used, yet laboratory results typically conformed to these unrealistic requirements for consistency; this could only have occurred as a result of the bias of laboratory technicians. 24In addition to affecting initial measurements, experimenter bias can also play a role in the way the data are managed and analyzed. Experimenters can include or exclude data based in part on experimenter expectations, and they can perform additional analyses, analyzing the data as raw scores or differences from baselines, and dividing subjects into any number of subgroups based on a range of different criteria. 25–27All of these procedures serve to drastically inflate the probability of detecting an expected effect by increasing the probability of producing a false-positive result.
We suggest that experimenter bias should receive more attention in laboratory research, and that blinding procedures to guard against experimenter bias should be a more common practice in laboratory investigations. Blinding procedures should be used when recording any measurements, not just in behavioral testing, and all decisions about inclusion/exclusion of data should be made before the blind is broken. We would also caution that some discussion or plan should be considered as to how the data will be analyzed before studies are completed, to reduce the risk that the rate of false-positive results will be inflated through the use of numerous unplanned analyses. Furthermore, we propose that the precise procedures used to prevent experimenter bias should be an essential component of the methods described in laboratory publications. Investigators should explicitly report the details of the procedures they used. A general statement that “the experimenter was blinded” is not sufficient. Finally, we also support the suggestion that laboratory studies should be evaluated in the same way that clinical trials are, with more emphasis on the difference between blinded and nonblinded studies and more weight given to studies that have carefully controlled for potential bias. 28
In summary, recent blinded experiments detected clear analgesic effects of spinal opioids in the spinal nerve ligation model. Previous, nonblinded experiments (one of them from our own laboratory) reported that spinal opioids were not effective for neuropathic pain, which was the expected result at that time. The differences in the results between these studies may have been attributed to experimenter bias in the earlier nonblinded experiments, a suggestion that is supported by the results of carefully blinded experiments conducted recently in our own laboratory. Experimenter bias is a well-recognized phenomenon in clinical trials, and although it does not receive as much attention in laboratory research, there should be no doubt that laboratory studies are just as vulnerable to experimenter bias as clinical studies. Thus, the very narrow focus of this editorial, on studies of spinal opioids in the spinal nerve ligation model of neuropathic pain, leads to consideration of a much more general issue—experimenter bias in laboratory research—and the suggestion that the use of rigorous blinding procedures may be just as important in laboratory experiments as they are in clinical trials.
