Visuospatial Ability as a Predictor of Novice Performance in Ultrasound-guided Regional Anesthesia

THE ability to perform practical procedures competently is essential to the safe practice of anesthesia. Ultrasound-guided regional anesthesia (USGRA) is a complex, invasive procedural skill that requires manual dexterity, hand–eye coordination, and a working knowledge of sonoanatomy.¹  International regional anesthesia societies have emphasized the need for training and competency assessment in USGRA to ensure safe practice.^2–4  However, some trainees will learn more quickly than others.⁵  Early identification of those who may require additional support is key to developing efficient expertise acquisition within the time constraints of postgraduate training.

Mental rotation is a visuospatial ability to mentally rotate and manipulate 2D and 3D objects. Mental rotation correlates positively with novice performance of simple laparoscopic tasks on benchtop models.^6–9  At a basic level, it is possible that laparoscopy and USGRA are similar with respect to the interaction of the operator’s hands and eyes with the ultrasound probe/laparoscope, the patient, and the screen. The importance of visuospatial ability has been emphasized in USGRA skills acquisition.¹  However, there is little evidence to support the use of visuospatial testing to identify individuals who may benefit from early, targeted training in USGRA. The primary aim of this study was to determine whether visuospatial ability could predict technical performance of an ultrasound-guided needle task by novice operators. Specifically, we chose to study the mental rotation test (MRT), the group embedded figures test (GEFT), and the Alice Heim group ability test (AH4).

Previous studies have considered the impact of visuospatial ability on novice skill performance in isolation. However, the role of traits and state emotional processes are also important for a fuller understanding of healthcare provision through their influence on clinical skills acquisition.^10,11  Specifically, previous studies have not investigated the interplay between skill performance and emotional processing.^1,12–14  Two emotional components are relevant here. State anxiety is well known to be related to performance.¹⁵  The trait, fear of failure, is a subclinical form of state anxiety when success is being valued.^16,17  Fear of failure could potentially hinder performance.^18,19  In addition, general cognitive ability is considered to be one of the best predictors of performance overall.²⁰  Therefore, we also aimed to study the relationship among emotional state, fear of failure, and intelligence with novice skill performance of an ultrasound-guided needle task.

The study was reviewed and approved by the University of Nottingham Medical School Research Ethics Committee (Approval Reference: L13092012 SCS Anesthesia). Medical students from the University of Nottingham Medical School were invited to participate in the study through poster advertising. Students who expressed a wish to participate were emailed a participant information leaflet and an invitation to attend the study. Written informed consent for the study, including video recording, was obtained from all participants.

This single-center, prospective, blinded observational study was conducted at the University Department of Anesthesia, Queen’s Medical Centre, Nottingham University Hospitals NHS Trust, Nottingham, United Kingdom. Subjects with previous experience of ultrasound scanning or performing regional anesthesia were excluded from the study. The study was organized in four phases (fig. 1). The enrolled medical students were asked to undergo and complete all four phases of the study. Participants’ identities were masked throughout the study, and their assessments were concealed from view within individual folders. Assessors of the ultrasound-guided needling task (phase 4) were blinded to the outcomes of the preceding assessments.

We collected basic demographic data including age, sex, year of study in medical school, and previous experience of USGRA.

This phase consisted of standardized visuospatial, emotional, and numerical reasoning assessments of the study participants. The assessments were paper based and administered under examination conditions, as per their standardization. Participants were blinded to the study hypothesis and their test scores. Brief descriptions of each visuospatial, emotional, and numerical reasoning assessment are described in the following text.

The visuospatial assessments consisted of the MRT, GEFT, and AH4.

There are four different variations of MRT: MRT-A, MRT-B, MRT-C, and MRT–D.^21–23  We used MRT-A, which consists of 24 problem figures. Each problem task has a target figure on the left and four stimulus figures on the right. Two of these stimulus figures are rotated versions of the target figure and two of the stimulus figures cannot be matched to the target figure. The aim is to mentally rotate the figures around the vertical axis to find the two correct rotated versions of the target figure. Participants were given 4 min to complete the first set of 12 problem tasks, followed by a 1-min break before completing the second set of 12 problem tasks in the next 4 min.

The GEFT measures field independence, which is the ability to perform a focal task independently of any background information or distracters.^24,25  The aim is to find a previously seen simple figure within a larger complex figure, which has been structured in a way to obscure or embed the simple figure. The participants were required to identify and outline accurately a simple shape embedded in a complex figure. The test consists of three sections. The participants were initially given 2 min to complete the seven problems in the first section. After this, second and third sections consisting of nine problems each were completed in 10 min.

The AH4 is designed as a group test of general intelligence, which primarily assesses deductive reasoning including verbal, mathematical, and spatial reasoning.²⁶  We used AH4 to assess spatial reasoning skills, which is the ability to visualize, mentally rotate, and manipulate two- or three-dimensional shapes or patterns. The test consists of 65 questions, and participants were given 10 min to complete as many questions as possible.

Emotional processes that tap into state anxiety or tense arousal (TA), as well as positive mood states (e.g., energetic arousal [EA]), were assessed using the UWIST Mood Adjective Checklist (UMACL) and fear of failure.

The UMACL is used to measure mood and comprises three bipolar scales: EA (vigorous vs. tired: coefficient α = 0.79), TA (nervous vs. relaxed: coefficient α = 0.76), and hedonic tone (pleasant vs. unpleasant mood: coefficient α = 0.81).^27,28  In addition to these scales, a monopolar anger/frustration (coefficient α = 0.80) scale was also used. The participants were instructed to complete the UMACL checklist according to their present mood using 28 adjectives each on a 4-point scale (“definitely”, “slightly,” “slightly not,” or “definitely not”).

Fear of failure assesses the general preference to be motivated not to succeed but to avoid failing.²⁹  This assessment consists of four statements pertaining to fear of failure, each scored on a 4-point scale (“always,” “often,” “rarely,” and “never”). Scores were then obtained by summating the item scores. The reported coefficient α was 0.70.

This test measures mathematical and logical reasoning via 20 short reasoning problems based on numbers that do not require any previous training in mathematics. It is a test of fluid intelligence, which depicts skills of problem-solving, abstract reasoning, and the ability to learn new things, irrespective of previous knowledge or education. There are 20 items, which include series completion (numbers and matrices), basic arithmetic problems (computational speed), and other deductive reasoning tasks. The participants were given 15 min to solve as many problems as possible.

In this phase, the participants were given 30 min to watch and review an 11-min video³⁰  mapped to specific learning objectives, which modeled expert performance of ultrasound-guided needle advancement in a turkey breast model.

The learning objectives were as follows:

1. Switch on the ultrasound machine (S-Series, Sonosite Limited, United Kingdom).

2. Correctly orientate the ultrasound probe (linear, 38 mm) in relation to the display on the screen.

3. Ensure adequate application of conducting gel to enhance ultrasound transmission and picture quality.

4. Locate and identify the target (olive) within the turkey breast.

5. Adjust the gain function to improve the quality of the image by altering brightness of the picture.

6. Alter the depth of the image to obtain a suitable image of the target.

7. By using an in-plane approach, insert a 50-mm Stimuplex^® A needle (B. Braun, Germany) into the turkey breast and aim to place the needle tip at the 12 o′clock position, as indicated by the attending assessors, above the upper edge of the target, without piercing the target.

The fourth phase included an ultrasound-guided needling task and its assessment. Participants were asked to complete the ultrasound-guided needling task, as demonstrated in the video, on a turkey breast model^31,32  using a standard ultrasound transducer probe (38-mm high-frequency linear array transducer; HFL38X 13-6 MHz, Sonosite Limited). To improve realism, the turkey breast model was inserted into the draped groin recess of a Laerdal^® IV Torso manikin (Laerdal Medical Limited, United Kingdom), which was used solely for this study. The participants received no help or feedback before or during the task. Study participation ceased once the ultrasound-guided needling task was completed.

Participants were independently assessed by two anesthesiologists experienced in USGRA as they performed the task. The assessors used two previously validated assessments of USGRA technical performance: composite error score (CES)^32–34  (appendix 1) and global rating scale (GRS)^35–37  (appendix 2). The assessors had undergone specific training and practice in the use of these assessment tools. The CES was calculated by adding the total number of errors, number of needle passes, and image quality score for each participant. A lower CES is associated with better accuracy and task performance. The GRS consisted of seven items each rated on a five-point scale. The GRS predominantly assessed more general behaviors and the overall performance of the participant.

Descriptive statistics for demographic and outcome measure data were calculated. CES data follow a nonnormal distribution and are thus presented as median (interquartile range). Normality of other data was assessed by histogram and the Shapiro–Wilk and Skewness/Kurtosis tests.

We performed an initial exploratory analysis using Spearman correlation coefficient ρ (rho) to determine which of the six explanatory variables was the most predictive of better task performance. The count data (CES) were overdispersed. This was unlikely because of excessive zeros as the proportion of extra zeros was considerably small (4/60 = 6.66%); therefore, negative binomial regression analysis was conducted for CES. For continuous data, the relationship between each potential explanatory variable and GRS was evaluated in an ordinary least square (OLS) regression. Bonferroni corrections were applied for multiple testing. We then created a regression model using the explanatory variable most predictive of performance. To examine which aspects of USGRA technical performance were most strongly correlated with visuospatial ability, we then deconstructed both assessments into their respective domains to perform a subanalysis with the predictive variable; this subanalysis was outside the previous validation of the CES and GRS. To achieve a study power of 0.8 (α = 0.05), we calculated that we would need to recruit 60 participants for this model with an assumed moderate effect size³⁸  (r = 0.3 to 0.5).

We chose to make nonpairwise comparisons between males and females to determine whether any differences in visuospatial ability existed. In all cases, we used P values less than 0.05 (two tailed) to indicate statistical significance.

Reliability of the assessment tools, i.e., CES and GRS, was evaluated using intraclass correlation coefficient, Cronbach α coefficient and SE of the measurement as a percentage of the mean (SEM [%]).^39,40  The statistical analysis software STATA/IC version 10.0 (StataCorp, USA) was used for data analysis.

All individuals who expressed an interest in participating in the study were recruited. Participant demographics and summary statistics for visuospatial ability and task performance are summarized in table 1. Males were found to exhibit better mental rotation skills compared with females (P < 0.001; table 2).

The intraclass correlation coefficient and SEM (%) for CES and GRS were 0.97 (15.29%) and 0.91 (8.53%), respectively; this demonstrates a high degree of interrater agreement. Similarly, Cronbach α coefficient and SEM (%) for CES and GRS were 0.98 (9.49%) and 0.96 (5.69%), respectively; this demonstrates a high degree of interitem consistency.

Of the three visuospatial assessments (MRT, GEFT, and AH4), only MRT correlated significantly with CES (ρ = −0.54; P < 0.001; fig. 2; table 3), indicating that a high error rate is associated with low MRT scores. The negative binomial regression coefficients for each variable showed that for each unit increase in MRT, the expected log count of the CES decreases by 0.08 unit (P < 0.001). After Bonferroni adjustments (P < 0.0016), only needle advancement without visualization of needle tip (ρ = − 0.52; P < 0.001) and number of needle passes (ρ = −0.45; P < 0.001) correlated significantly with MRT.

Of the three visuospatial assessments (MRT, GEFT, and AH4), only MRT correlated significantly with GRS (ρ = 0.47; P < 0.001; fig. 3; table 3), indicating that better performance was associated with better mental rotation skills. An OLS regression established the univariate association of GRS with MRT showing that for a 1-unit increase in MRT, we would expect a 0.43-unit increase in GRS (P = 0.002). After Bonferroni adjustments (P < 0.0031), only time and motion (ρ = 0.44; P < 0.001), instrument handling (ρ = 0.47; P < 0.001), and flow of procedure (ρ = 0.44; P < 0.001) correlated significantly with MRT.

Of the UMACL, EA was found to correlate negatively with CES (ρ = −0.30; P = 0.01; table 3. By contrast, TA correlated positively but weakly with CES (ρ = 0.26; P = 0.04; table 3. This showed that errors would be low in vigorous, active individuals and high in anxious individuals. The negative binomial regression coefficients for each of the variable showed that for each unit increase on EA, the expected log count of the CES decreases by 0.07 unit (P = 0.02).

Of the UMACL, only TA correlated negatively with GRS (ρ = −0.35; P = 0.005; table 3). This showed that GRS quality scores would be low when anxiety level is high. An OLS regression established the univariate association of GRS with TA showing that for a 1-unit increase in TA, we would expect a 0.54-unit decrease in GRS (P = 0.01).

The results indicate that MRT predicts technical performance of an ultrasound-guided needle advancement task by novice operators. The study shows that males are likely to have better mental rotation capabilities than females. This is in line with two previous meta-analyses,^23,41  which showed effect sizes around 0.95 favoring males. This difference means that the MRT cannot be used as a selection tool for medical posts, i.e., high stakes assessment, because men are likely to be preferentially selected, and this would introduce indirect sexual discrimination against women. However, that is not to say that men with better MRT scores perform better at the task. Two previous studies of laparoscopic skills have demonstrated that gender does not affect psychomotor performance,^42,43  although the affect of MRT scores of both males and females remained unknown. In an observational study of surgical trainees with very limited laparoscopic experience, Grantcharov et al.⁴³  demonstrated that male trainee surgeons made a similar number of errors and unnecessary hand movements during their performance of six simulated laparoscopic tasks similar to females.

One may suggest that it would be useful to provide a range of MRT scores wherein learners could benefit the most from focused training. However, we are unable to do so at this stage of our work. Although our study shows that MRT has a predictive validity for performance of an ultrasound-guided needle task, it does not indicate at what point MRT performance can be defined as adequate. Despite this limitation, we believe that it is reasonable to state that low error rates, better image quality, and better global performance are associated with higher MRT scores. Therefore, strategies that aim to develop mental rotation skills could be developed and used to improve ultrasound-guided needle advancement skills.

We also found that negative mood adversely affects performance. Therefore, good performance seems to be a function of visuospatial ability and reducing anxiety. Thus, a second practical implication of our study is that stress and anxiety when performing the task may need to be reduced in the training and learning environment. However, in the real clinical setting, some level of anxiety will be associated with any clinical intervention, and thus, the degree of stress elicited by the task may itself increase its validity.¹⁰  It may be more relevant for educators to develop curricula that allow novice practitioners to learn the necessary coping skills to deal with the emotional costs of this type of work and procedure.

Limitations are integral to any investigation and warrant specific comment here. Aside from the ethical problems of allowing novice practitioners to practice on real patients, it is likely that anatomical differences, doctor–patient interactions, and the pressures of achieving successful blocks would generate inconsistent results in real clinical situations.^1,33  For the purpose of this study, we used a turkey breast benchtop model rather than in vivo needling. Despite the lack of clinical context, we believe that our participants experienced an “examination-like” stress caused by their assessments during the study. In addition, “live assessment” of the participants may have added to their stress levels. It is likely that the combined stress and fear could produce detrimental and variable effects on performance such as that in clinical practice.^15,44,45  Despite the limitations of the turkey breast benchtop model, it is accepted as an initial means to evaluate novice performance in USGRA³  and to perform training in this complex technical task.³¹  As such, we believed that our benchtop simulation provided a reproducible and realistic environment in which to assess our subjects.

The subjects were medical students and not practicing doctors; therefore, one could argue that with seniority and experience, there is an inherent level of psychomotor expertise, which confers an improved ability to perform new psychomotor skills. Thus, it could be debated that had we studied anesthesiology residents, the correlations between MRT and GRS or CES may have been weaker. However, previous work has demonstrated that medical student’s performance of an ultrasound-guided needle task is broadly comparable with that of novice resident doctors.^32,33  Hence, we do not consider that the use of medical student volunteers presents a significant limitation to our study; rather the fact that they have no experience may be considered a positive aspect of the study.

We have used the assessments of visuospatial ability, emotional processing, and general cognitive ability, which are considered to be valid and reliable.^21–27  With regard to CES and GRS, we have demonstrated high levels of interrater agreement and internal consistency of the assessment tools; this is in line with the previous findings.^33,37  We believe that the high interrater agreement reflects the assessor training with the CES and GRS tools before recruitment. As such, we believe that our measurement of task performance is both reliable and valid.

Finally, we have attempted to mitigate for any bias in assessment by asking our assessors to rate the participants’ performance independent of one another and without knowledge of the participants’ scores in the various psychologic assessments completed beforehand. The premise of this study was to identify whether visuospatial ability could be used to predict technical performance of an ultrasound-guided needle task by novice operators. In doing so, we have identified correlations among performance, mental rotation skills, and negative mood. As a trait measure, MRT has the potential to be used as a tool to focus educational and training resources on individuals who have less ability to perform ultrasound-guided needle tasks.

Future research could investigate whether specific training interventions could transform visuospatial ability and thus enhance skills acquisition in USGRA. In broader terms, we believe that the predictive value of MRT in videolaryngoscopy and fibreoptic intubation should be investigated. With regard to MRT as a screening tool to focus training, we believe that future studies need to assess the sensitivity and specificity of MRT in this context.

This study was supported by an educational grant from Regional Anesthesia—United Kingdom, London, United Kingdom.

Dr. Hardman is an editor of British Journal of Anesthesia. Dr. Ferguson was a principal investigator on a case award grant from the ESRC (Integrating Prospect theory [Framing Effects] and the Common Sense Model of illness to improve medication compliance in glaucoma patients); was a coinvestigator on a grant from Chief Scientists Office, Scotland (A randomized controlled trial to test if a simple anticipated regret manipulation leads to a significant increase in organ donor registrations); is a joint PhD supervisor on a project with Experian (Potential of Psychological Information to Inform Credit Scoring); is a coinvestigator on a grant from DEFRA (Overcoming Barriers to Uptake of Best Welfare Practice by Sheep Farmers); is a coinvestigator on a grant from Pfizer (Pain Phenotypes in Rheumatoid Arthritis: A Patient-centered, Stratified Approach to Optimizing Disease Modifying and Analgesic Treatments); is a coinvestigator on a grant from Arthritis UK Research (Pain Centre); supervises externally funded PhD students (ESRC PhD studentships; Reluctant Altruism, Schizophrenia and Empathy); was paid as an Associate Editor for Journal of Behavioral Medicine and as Deputy Editor for British Journal of Psychology; is paid as an Associate Editor for Annals of Behavioral Medicine; is an invited speaker at ISBT (expenses and travel paid); is paid as an external examiner at Edinburgh University and Lancaster University; has had expenses paid to present talks at (1) Workshop: Right Patient, Right Place, Right Time: When and How to Use the Emergency Department: Health Literacy, Decision-making and Communication, Unversita Cattolic del Sacro Cuore, Milan, Italy, April 11 to 12, 2014, (2) Taxometrics, Psychometrics and Clinometrics: Stratified Medicine and Assessing Phenotypes—Economic and Social Research Council (ESRC) Advanced Quantitative Methods National Conference, White Rose DTC, Sheffield University, Sheffield, July 24 to 25, 2014, and (3) Personality: Myths, Misconceptions and Misunderstandings—Implications for Current and Future Medical Education, Selection and Training, INReSH London, November, 10 to 11, 2014; receives royalties each year from GL Assessment for the Pediatric Index of Emotional Distress; is a coinvestigator on a grant from the U.S. Defense Medical Research and Development Program (Study to Examine Psychological Processes in Suicidal Ideation and Behavior), which does not relate directly to the current submitted work but is a different grant; he is a coinvestigator on a grant from the ESRC (Individual Differences in the Impact of Socio-Economic Events on Health and Well-being) and this does not relate directly to the current submitted work but is a different grant. Dr. Bedforth runs and is a faculty member on teaching courses sponsored by Sonosite Ltd. and NIHR National Institute for Health Research—Research for the Patient Benefit. The other authors declare no competing interests.