Factors Affecting Admission to Anesthesiology Residency in the United States: Choosing the Future of Our Specialty

• Although the change in number of applicants to U.S. anesthesia residencies has been quantified, a detailed examination of factors associated with successful matching in these programs has not been performed.

• In a review of 2010 to 2011 residency applicants to Northwestern University, the odds of a successful match at any U.S. residency was greater (2.8) for U.S. graduate applicants younger than 29 yr old, and 1.2 for female applicants. The average applicant had no peer-reviewed publications.

• These data, representing approximately half of all applicants to U.S. anesthesia residencies, suggest a strong bias against older applicants, a slight bias in favor of women, and lack of research publications in successful applicants.

MATCHING and matriculating in an anesthesiology residency in the United States has become much more competitive than 5 yr ago. According to the National Residency Matching Program, only 36% of anesthesiology residency spots were filled by U.S. medical students in the 1995 match. In contrast, 87% of spots offered in the 2010 match were filled by U.S. medical students**(fig. 1). The high demand for medical students to pursue an anesthesiology residency creates a unique opportunity for our specialty. Academic programs can now select better applicants who will be more capable to further advance the science and the various needs of our profession.

Other traditionally competitive specialties, such as dermatology and pediatric surgery, have examined factors associated with a successful admission to their residency training programs.1,2Factors associated with a successful admission to anesthesiology residency have not been quantified. The identification and importance of factors associated with a successful residency match is valuable for both applicants (they can better prepare for a competitive journey) and academic programs (they can compare their values with national standards).

A critical need for more anesthesiologist-researchers has been recognized in order to advance the specialty as a respected member of the academic community.3Since the interest for research usually develops early in the medical career,4one of the pathways proposed to respond to that critical need was to recruit research-oriented medical students by residency programs.5It has not been determined if applicants with prior research productivity are more likely to be admitted to an anesthesiology residency than applicants with no research productivity.

Residency selection in the United States is also based on subjective criteria during the interview process, such as an evaluation of applicant's interpersonal skills and his or her interaction with program faculty. Any process that involves selection of personnel based on subjective criteria during personal interviews is vulnerable to the developments of demographic bias (age, gender, and race) because of the subjective rating of candidates.6Other postgraduate disciplines have demonstrated or refuted the effect of demographic bias in the selection process of its residency applicants.7,8,9It is also unknown if the selection of anesthesiology residents is affected by these demographic factors.

The objective of this current investigation is to examine factors affecting a successful admission to an anesthesiology residency in the United States.

The study was a retrospective cohort investigation. The protocol was approved by the Northwestern University Institutional Review Board, Chicago, Illinois. The requirement of informed consent for this study was waived by the Board.

The data were obtained from the Electronic Residency Application Service. The Electronic Residency Application Service was developed by The Association of American Medical Colleges to electronically transmit application materials from medical schools to residency programs. During the 2010 and 2011 admission cycle periods, applicants could apply to 132 anesthesiology programs. Programs evaluate applications and invite applicants to interview. Applicants and programs submit their rank lists to the National Residency Match Program, which utilizes an algorithm to match individuals with residency positions. Highly ranked applicants by programs are more likely to be admitted to a residency program.

The data from applicants to Northwestern University's anesthesiology residency program for the application periods of 2010 and 2011 were examined. The data were de-identified in order to protect applicant's privacy. The data were extracted by two of the investigators (TA and MCK). A typical applicant applies to more than 20 programs across the country, which enables access of a single program to obtain a large national sample of applicants.††

Data extracted from the applications included applicant demographics' information (age, gender, ethnicity) and objective data used by programs to select residents including United States Medical Licensing Examination (USMLE) scores for Step 1 and 2, medical school origin (United States vs.  international), membership in Alpha Omega Alpha Honor Society, medical school class rank, graduate school education (applicants who received either a master's degree or a Ph.D.), and the presence and number of peer-reviewed publications in the applicant's curriculum. The primary outcome was a successful anesthesiology residency match. A successful match was defined as the applicant who matched and was admitted to an anesthesiology residency program. An unsuccessful match was defined as the applicant who did not match to an anesthesiology residency program.

Assuming a 85% matching success, a sample of 1,000 applications would be required in order to evaluate 10 predictors in the nonmatched group, assuming a rate of 15 events to 1 predictor.10 

A conditional inference tree analysis was constructed to model a decision tree for matching in an anesthesiology residency program. The conditional tree algorithm uses binary recursive splitting to classify applicants belonging to either the matched or unmatched groups. Because 10 variables were evaluated for recursive splitting, a P < 0.005 was selected as the minimal criterion for splitting of the groups. Stopping criteria for the analysis is based on multiplicity-adjusted P  values with Bonferroni correction. Based on the primary division of the conditional tree analysis, characteristics of the applicants who successfully matched in an anesthesiology resident program were compared with applicants who did not match using a Fisher exact test or the Mann–Whitney U test. The association between predictor variables was evaluated using Spearman rank test. The overall predictive value of the conditional tree model was assessed as the area under the receiver-operator characteristics curve of a successful match predicted by the model. Sensitivity, specificity, positive and negative predictive values, and the positive likelihood ratio of a test positive were calculated using standard formulae. To evaluate the effect of age bias in the selection process, we performed a propensity score-matched analysis. The propensity score was the conditional probability for age 28 yr or younger or more than 28 yr, as a binary dependent variable, under a set of predetermined covariates, added into a multiple logistic regression. Each applicant got an individualized propensity score derived from the logistic regression. Using these estimated propensity scores, we performed a one-to-one matched analysis (nearest neighbor with caliber matching) and randomly selected an applicant with age 28 yr or younger versus  applicants with an age more than 28 yr who had the closest estimated propensity score. The pair of applicants would be eligible for matching if the caliber width of pair is within 0.6 SD of all selected pairs. Cochran and Rubin had suggested that a caliber width of 0.6 SD will remove approximately 90% of the bias in observed confounders.11Applicants who did not have an acceptable range of match were excluded. Logistic regression analysis was then performed to estimate the odds ratio for the propensity-matched groups. Significance was accepted at P = 0.005. Data were analyzed using Stata version 11 (StataCorp LP, College Station, TX) and R version 2.14.0, release date October 31, 2011 (The R Foundation for Statistical Computing, Vienna, Austria).

For this study, 1,976 applicants for anesthesiology residency for the admission years of 2009 to 2010 and 2010 to 2011 were evaluated. There were 1,773 applicants competing for 1,385 residency spots ranked anesthesiology in the 2010 year, and 1,748 applicants competed for 1,404 residency spots in the 2011 year.3Data from the 1,976 applicants represented 58% of the total national applicant pool. The mean age of applicants was 29 ± 4; 64% were males and 36% females; and 51% were Caucasians. The median (99.5% CI) number of publications per applicant was 0 (0 to 1). Among the 1,976 applicants evaluated, 66% percent successfully matched to an anesthesiology residency position. Mean USMLE Step 1 and 2 scores for the 1,976 subjects were 222 ± 19 and 229 ± 19, which were not different from the 2011 match cycle Step 1 (222 ± 17) and Step 2 (229 ± 19) scores from the national sample. ‡‡

U.S. medical school graduates, age, USMLE Step 1 and 2 scores, and gender were identified by conditional inference tree analysis for classification of applicants for successful match into an anesthesiology residency program (fig. 2). The odds (99% CI) for a successful match for a U.S. medical school graduate was 3.6 (3.1–4.2) compared with an international medical school graduate, 2.6 (2.3 to 3.0) for applicants with USMLE Step 2 scores more than 210, and 1.2 (1.1 to 1.3) for female applicants. At a cutoff P = 0.5, the conditional inference tree accurately (99.5% CI) classified 0.81 (0.76 to 0.85) of the applicants. The sensitivity and specificity (99.5% CI) of the classification was 0.87 (0.82 to 0.92) and 0.69 (0.61 to 0.78), respectively. The positive predictive value of a test positive was 0.84 (0.79 to 0.89), the positive likelihood ratio was 2.8 (2.1–3.1), and the negative predictive value was 0.73 (0.65 to 0.82).

Younger age, higher USMLE Step 1 and 2 scores, female gender, and ranking in the upper third of the applicant's medical school class were associated with a successful match among international medical school applicants (table 1). Within this subset, older age was correlated with lower USMLE Step 1 scores (ρ = −0.24, P < 0.0005) and USLME Step 2 scores (ρ = −0.15, P < 0.001) but gender did not correlate with any other applicant characteristics. Compared to applicants that did not match, U.S. medical school applicants with USMLE Step 2 scores more than 210 that matched were younger and had higher scores on USMLE exam Step 1 and 2. Females were more likely to match than male applicants. (table 2). Within this subset, older age was correlated with lower USMLE Step 1 (ρ = −0.15, P < 0.0005) and Step 2 scores (ρ = −0.12, P < 0.0005) and with previous graduate education (ρ = 0.24, P < 0.0005), but gender did not correlate with any other applicant characteristics. U.S. medical school applicants with USMLE Step 2 scores less than 210 were further classified by gender. Applicants who matched were younger and had higher USMLE scores compared with those who did not match (table 3). Within this subgroup, older age was associated with previous graduate education (ρ = 0.21, P < 0.002), but age and gender were not correlated with any other applicant characteristics.

Propensity-adjusted samples for applicant's age more than 28 yr among international and U.S. medical school applicants are shown in tables 4and 5. Propensity-matching logistic regression found the odds (99% CI) for a successful match for applicants younger than 29 yr was 3.3 (2.0–5.4) and 2.8 (1.9–4.2) for international and U.S. graduates, respectively.

We could find no evidence of preference for applicants with a history of graduate education and peer-reviewed publications. Of the U.S. medical school applicants younger than 28 yr, 40% had scholarly activity compared with 46% of those more than 28 yr old (P = 0.03). Of the U.S. graduates older than 28 yr who matched, 45% had peer-reviewed publications compared with 48% of those who did not match (P = 0.58). In the less than 28 yr age group, 40% of those who matched had peer-reviewed publications compared with 37% of those who did not match (P = 0.43). Of the U.S. medical school candidates with no prior graduate education, there was no difference in the proportion with research productivity among the matched 39% and the unmatched 40% (P = 0.5). The correlation between the number of publications and match in the top 20 National Institutes of Health-funded institutions was ρ = 0.04, P = 0.59 for international medical school applicants and ρ = −0.06, P = 0.12 for U.S. medical school applicants. Overall, the association was ρ = −0.08, P = 0.03.

Applicants included in this study matched in 125 different residency programs. Of the 679 nonmatched applicants, 80 had matched in previous residences and 28 of the 297 U.S. candidates who did not match had attended graduate school. Sixty-seven applicants matched in a specialty other than anesthesiology, but only 19 matched in more competitive residency programs based on 2010 match data. Thirty-six percent of the applicants matched at a top 20 research institution in the U.S.

The important finding of the current investigation is that U.S. medical school graduation and USMLE Step 2 board scores were the primary determining factors associated with a successful match and admission to a U.S. anesthesiology residency. Another important finding of our study was the presence of demographic bias with respect to age and gender in the selection of applicants. The bias toward the selection of female applicants was seen in international medical school applicants as well as those from U.S. medical schools, especially among U.S. medical school applicants who had lower USMLE scores. The bias observed against applicants older than 28 yr could somewhat be accounted for by the inverse relationship between age and USMLE board scores; however, despite propensity matching of subjects for other predicators, we were still able to observe an association of age on the likelihood of a successful match.

Applicants who were international medical graduates were less likely to successfully match than applicants who graduated from a medical school in the United States. This phenomenon has happened across all medical specialties and can be more accentuated as the specialty becomes more competitive.12Causes previously cited for a lower admission of international medical graduates include lower scores on the USMLE, medical schools unfamiliar to program directors, language barriers, and problems with visa requirements.13,14,15Specifically in anesthesiology, attending an international medical school has been associated with a higher failure on the American Board of Anesthesiology specialty exam.16Some individual programs may prioritize USMLE Step 1 instead of USMLE Step 2 scores in their admission process; however, USMLE Step 1 was only a distant branch of the decision tree analysis for American medical school graduates in the current investigation.

The other primary association with successful admission to an anesthesiology residency was higher scores on the USMLE Step 2 examination. This was also among one of the most important considerations for ranking applicants according to the postmatch survey of program directors conducted by the National Residency Match Program (table 6). Higher USMLE scores have not been associated with a better clinical performance during residency, but they have been associated with higher scores during residency in-training exams.17,18,19Programs where residents do well on in-training exams have been granted longer periods of accreditation.20Program directors have, therefore, an incentive to recruit applicants who do well on exams.

A bias toward the acceptance of a greater proportion of female applicants was not unexpected. A lower rate in the increase of female anesthesiology residents compared with female medical students has been previously demonstrated.21Our findings suggest that the low proportion of female residents in anesthesiology is more likely a result of a lower application rate by females to anesthesiology residency programs rather than a lower selection rate of female applicants. The low number of female applicants to anesthesiology residency might be an incentive to programs to accept female applicants in order to increase diversity.

Younger age was as an independent predictor of a successful match to anesthesiology residency. The effect of age on applicant selection was further confirmed by propensity-matching graduates from both international and U.S. medical school programs. The high cost of medical education in the United States can be contributing factor for some applicants to apply for a residency position at a later age.22Previous studies have not identify applicant's age as a predictor of poor residency performance in anesthesiology and other specialties.23,24,25 

Contrary to what was expected, there was a lack of an association between the presence of an applicant's peer-reviewed scholarly production and a likelihood of admission to anesthesiology residency. The typical applicant had no peer-reviewed publications in their application, and applicants who had attended a graduate school, which frequently involves some degree of research training, were not more likely to be admitted for an anesthesiology residency. The lack of emphasis on research experience/productivity in the selection of prospective anesthesiology residents was further confirmed by a postmatch survey of anesthesiology program directors who reported involvement and interest in research as one of the least important factors in ranking applicants (table 6). Although other competitive specialties have valued research skills among their applicants,1,2anesthesiology seems to be missing an important window of opportunity to meet the current/future needs of the specialty.3,5We suggest that a requirement for research productivity during residency training by the Accreditation Council for Graduate Medical Education will create an incentive for programs to recruit applicants with prior research experience and productivity. Higher commitment to research by the anesthesiology specialty leadership has been cited as a critical factor in changing the current need to develop research within the anesthesiology specialty.3It is important to note that the selection of residency candidates who will have productive academic careers is a very complex process and candidates who have prior scholarly productivity may not achieve a successful scholarly career after residency graduation, even for those with prior master's degree or Ph.D. Nonetheless, other specialties have emphasized prior publication as a criteria for residency admission.

The presence of demographic bias in the selection of applicants raises questions regarding the need for the development of objective scoring systems by academic programs before submission of a ranking list to the match system.26This strategy may limit the possibility of bias regarding demographic factors but it is unlikely that it will eliminate all biases because of the strong emphasis that subjective criteria play in the selection of anesthesiology residents (table 6). Many countries around the world have a similar residency admission process to the United States, which involves both objective and subjective evaluations, making the process vulnerable to the development of demographic bias (table 7). Countries that perform resident selection based solely on objective information such as test scores avoid the presence of demographic bias on their resident selection process. Although this strategy may help restrict demographic bias during the selection process, it also limits the ability of programs and applicants to interact and evaluate their mutual expectation before establishing a commitment for the next years. The interview process is therefore not only important for academic programs but also for applicants making their choices.

Our study is only valid when interpreted within the context of its limitations. Despite the high capacity of our model to predict a successful match, we did not include variables that are often considered by programs in the selection of applicants, such as letters of recommendation and personal statements. The lack of standardized and validated instruments to evaluate those variables and the subjective nature of their assessment were the reasons why we decided not to include them in the study design. Recent evidence also suggests that personal statements discuss a number of common themes and thus might appear to offer little utility in differentiating residency candidates.27Although our sample size represented 58% of a whole national sample of applicants, it is possible that programs located in different geographic regions would have slightly different results than what we observed in our sample. The strong association between the independent factors (age, gender, USMLE scores, and origin of medical school) and a successful anesthesiology residency matching makes unlikely that small variations in the sample characteristics would lead to a different result than the one demonstrated in the current study.

In summary, we demonstrated that graduation from a medical school in the United States, USMLE Step 2 scores, younger age, and female gender were characteristics associated with a successful match to anesthesiology residency. Peer-reviewed research productivity was not associated with a successful admission to an anesthesiology residency. The presence of demographic bias (age and gender) in the selection of applicants suggests that academic programs examine their selection process and implement measures in the generation of their ranking list of applicants to avoid this phenomenon in the future. The poor peer-reviewed scholarly production of applicants to anesthesiology when compared with other competitive specialties, as well as the lack of emphasis on research requirements by academic programs during the admission process of anesthesiology residents, may be a concern for the academic future of our specialty.

The authors thank Ramon Andrade de Mello, M.D., Resident Physician, Instituto Portugues de Oncologia do Porto, Porto, Portugal; Marcus Pantaroto, M.D., Resident Physician, Instituto Portugues de Oncologia do Porto; Ling Hu, M.D., Assistant Professor, Northwestern University, Chicago, Illinois; and Ray Chang, B.S., Medical Student, Northwestern University, for their assistance in determining resident selection process across different counties.