To the Editor:—  Studies that use propensity analysis, like the study of Vincent et al. ,1should not be perceived to be inferior to the gold standard of prospective randomized studies. Rather, propensity analysis and prospective randomized studies should be interpreted as complementary methods for finding the truth. Despite Nuttall and Houle’s assertion that randomized controlled studies, unlike propensity analyses, do not have “the limitation that remaining unmeasured confounding variables may still be present,”2both measured and unmeasured confounding variables may still be present. Randomized studies rely on the assumption (or hope) that these variables will be equally distributed between the groups. Who the anesthesiologist is or who harvests the saphenous vein may have a profound effect on outcome after cardiac surgery,3,4but random studies involving cardiac surgery rarely stratify by these factors or even measure them. Even small differences between groups in measured variables in randomized trials may lead to erroneous statistically significant outcomes.5 

Prospective randomized studies may be limited by the inability to randomize for important variables. In evaluating an intervention, such as activated protein C on mortality of intensive care unit patients, it is necessary that nonrandom but important factors, such as which intensive care unit treats the patient, be controlled. Typically, this is done with severity scores such as the Acute Physiology and Chronic Health Evaluation and Mortality Probability Model. Although the word propensity  is not used to describe the Acute Physiology and Chronic Health Evaluation or Mortality Probability Model, these scores are the likelihood or the propensity that a patient will die, and these scores are then included (the same as a propensity score determining the likelihood of receiving a transfusion would be included in a study of blood transfusion and sepsis1) in the analysis to partially control for some of the confounders in the randomized controlled trial.

Another limitation of randomized controlled trials is their lack of generalizability. In determining the benefits or harm of transfusion, Hébert et al.  6evaluated 6,451 persons to randomize 838 subjects (13%); 5,613 patients were excluded from their study. Physician belief in equipoise, the patient’s or family’s beliefs, or excluding patients based on age or comorbidities may produce nonrepresentative populations in randomized trials and severely limit the generalizability of the results.7–9In addition, crossover of subjects from one arm to the other arm of the trial or subject withdrawal may make the results hard to interpret.

Observational studies are not necessarily inferior to randomized studies. Both have advantages and disadvantages. Observational studies should be encouraged as a complement to randomized studies. They include a greater variety of patients, many of whom would be excluded by randomized studies, and can be performed for a small fraction of the cost. Sophisticated and innovative statistical techniques, such as multivariable analysis, propensity, and instrumental variables10should be used to help separate gold from fool’s gold.

University of Toledo College of Medicine, St. Vincent Mercy Medical Center, Toledo, Ohio.

Vincent J-L, Sakr Y, Sprung C, Harboe S, Damas P: Are blood transfusions associated with greater mortality rates? Results of the Sepsis Occurrence in Acutely Ill Patients study. Anesthesiology 2008; 108:31–9
Nuttall GA, Houle TT: Liars, damn liars, and propensity scores. Anesthesiology 2008; 108:3–4
Slogoff S, Keats AS: Does perioperative myocardial ischemia lead to postoperative myocardial infarction? Anesthesiology 1985; 65:107–14
Poston RS, Gu J, Brown JM, Gammie JS, White C, Nie L, Pierson RN III, Griffith BP: Endothelial injury and acquired aspirin resistance as promoters of regional thrombin formation and early vein graft failure after coronary artery bypass grafting. J Thorac Cardiovasc Surg 2006; 131:122–30
Lee JL, McNeer JF, Starmer CF, Harris PJ, Rosarti RA: Clinical judgment and statistics: Lessons from a simulated randomized trial in coronary artery disease. Circulation 1980; 61:508–15
Hébert PC, Wells G, Blajchman MA, Marshall J, Martin C, Pagliarello G, Tweeddale M, Schweitzer I, Yetisir E: A multicenter, randomized, controlled clinical trial of transfusion requirements in critical care. Transfusion Requirements in Critical Care Investigators, Canadian Critical Care Trials Group. N Engl J Med 1999; 340:409–17
Sprung CL, Eidelman LA: Ethical issues of clinical trials for the pulmonary artery catheter. New Horiz 1997; 5:264–7
Sen Biswas M, Newby LK, Bastian LA, Peterson ED, Sugarman J: Who refuses enrollment in cardiac clinical trials? Clin Trials 2007; 4:258–63
Britton A, McKee M, Black N, McPherson K, Sanderson C, Bain C: Threats to applicability of randomised trials: Exclusions and selective participation. J Health Serv Res Policy 1999; 4:112–21
Newhouse JP, McClellan M: Econometrics in outcomes research: The use of instrumental variables. Annu Rev Public Health 1998; 19:17–34