Comparison of the TOFscan and the TOF-Watch SX during Recovery of Neuromuscular Function

COMPLETE recovery of neuromuscular function (train-of-four ratio greater than or equal to 0.9) should be present at the time of tracheal extubation in order to enhance patient safety and improve quality of recovery in the early postoperative period.^1,2  However, significant levels of neuromuscular blockade may still be present when standard clinical testing indicates muscle strength has fully returned. Studies in awake volunteers have documented that the majority of subjects can maintain a 5-s head-lift and open their eyes at train-of-four ratios as low as 0.5.^3,4  Furthermore, clinicians are unable to reliably detect fade using peripheral nerve stimulators at train-of-four ratios above 0.4.^5,6  In order to assess readiness for tracheal extubation, more precise objective monitors are required.⁷  However, surveys suggest that clinicians infrequently assess neuromuscular function with quantitative devices.^8,9  Several factors may account for this reluctance to use these monitors, which include the need for additional operating room time (attachment of sensors, initial signal stabilization), requirement for calibration and obtaining baseline measurements before muscle relaxant administration, and lack of accuracy and precision of first-generation quantitative monitors when an initial set-up is not performed.

At this time, only acceleromyography devices are commercially available as stand-alone quantitative monitors. Acceleromyography technology uses a piezoelectric sensor to measure acceleration of a muscle or digit. Acceleration of a freely moving muscle or digit generates voltage in the piezoelectric transducer that is analyzed and displayed on a monitor. First-generation acceleromyography monitors employed a sensor that measured acceleration in a single plane. A new acceleromyography monitor, the TOFscan (Drager Technologies, Canada), uses a three-dimensional piezoelectric sensor that attaches to the thumb via a hand adapter (adult and pediatric sizes) and measures acceleration in multiple planes. The TOF-Watch SX acceleromyography device (Organon, Ireland) is currently considered the accepted standard monitor for use in clinical trials.¹⁰  The aim of this investigation was to assess the agreement between train-of-four ratios obtained with the TOFscan and those obtained with the TOF-Watch SX during spontaneous recovery of neuromuscular function in surgical patients who have been administered rocuronium. We hypothesized that good agreement would be observed between the TOFscan and the TOF-Watch SX when calibration, preload, and normalization were applied to the TOF-Watch SX.

This prospective, observational clinical trial was approved by the NorthShore University HealthSystem Institutional Review Board (Evanston, Illinois) and registered at ClinicalTrials.gov (NCT02433769; principle investigator Glenn Murphy, registration date April 23, 2015, patient enrollment June 25, 2015 to February 18, 2016). The study was conducted at a single tertiary medical center (NorthShore University HealthSystem, Evanston, Illinois), and written informed consent was obtained from all subjects.

A total of 25 American Society of Anesthesiologists Physical Status I to III patients, ages 18 to 80 yr, presenting for elective surgical procedures with an expected duration of more than 90 min, were enrolled in the investigation. Exclusion criteria included: need for succinylcholine for rapid sequence intubation; presence of renal insufficiency (defined as a serum creatinine concentration greater than 2.0 mg/dl) or renal failure; significant liver disease (cirrhosis or hepatic failure); presence of neuromuscular disease; use of drugs known to interfere with neuromuscular transmission; obesity (body mass index greater than or equal to 30 kg/m²); or surgical procedures in which maintenance of neuromuscular blockade was required or access to both arms was not possible.

Anesthetic management was standardized for all patients. Monitoring consisted of an electrocardiogram, a noninvasive blood pressure cuff placed on the opposite side of the intravenous line (inflations were stopped for at least 5 min after neuromuscular blocking agent administration to ensure plasma concentrations of rocuronium were similar in both arms), capnography, central temperature monitoring (nasopharyngeal or esophageal), a Bispectral Index monitor (BIS system; Aspect Medical Systems, USA), and a peripheral nerve stimulator. Anesthesia was induced with propofol 2.0 mg/kg, lidocaine 50 mg, fentanyl 100 μg, dexamethasone 8 mg, and rocuronium 0.6 mg/kg. Clinicians were allowed to administer additional rocuronium, with the objective of achieving complete spontaneous neuromuscular recovery at the conclusion of surgery. Anesthesia was maintained with sevoflurane 1.0 to 3.0%, with the concentration adjusted to maintain the Bispectral Index between 40 and 60, and systemic blood pressure within 20% of baseline values. Hypotension was treated with ephedrine (5 mg), phenylephrine (80 μg), or a fluid bolus, as clinically indicated. Hypertension was treated by increasing the concentration of sevoflurane. Additional doses of fentanyl were administered as needed, and hydromorphone was given at the conclusion of surgery when required. End-tidal carbon dioxide values were maintained between 30 and 34 mmHg. Core temperatures greater than 35.0°C and upper extremity temperatures greater than 32.0°C were achieved with upper body forced-air warming devices (Bair Hugger; Augustine Medical, USA). Ondansetron 4 mg was given 30 min before the conclusion of surgery. Neostigmine 40 μg/kg and glycopyrrolate 8 μg/kg were administered at the end of surgery to ensure complete recovery of neuromuscular function in patients who potentially did not spontaneously recover to a train-of-four ratio greater than or equal to 0.9. Clinicians were blinded to train-of-four data, and tracheal extubation was performed on the basis of clinical criteria alone (sustained head-lift for greater than 5 s, the ability to follow simple commands, stable ventilatory pattern with an acceptable arterial pulse oximetry reading).

Neuromuscular monitoring was applied 10 to 15 min before induction of anesthesia and was conducted as recommended in the Good Clinical Research Practice guidelines.¹¹  The TOFscan and TOF-Watch SX devices were applied to opposite arms (figs. 1 and 2). After careful abrasion and cleansing of the skin, two surface electrodes were placed over the ulnar nerve on each arm, with the negative electrode near the wrist and the positive electrode 3 to 4 cm proximally. On the hand with the TOF-Watch SX, a preload device was attached (Hand Adapter; Organon) that applied a constant preload of 75 to 150 g. The acceleration transducer of the TOF-Watch SX was placed in the Hand Adapter, and the Hand Adapter was attached to the hand using an elastic band and adhesive tape. On the arm with the TOFscan, the thumb splint with the encased transducer was secured to the hand with tape. The arm and fingers were secured on the armboards of the operating room table using Velcro straps to prevent movement of the fingers during nerve stimulation.

Neuromuscular monitoring was initiated after anesthetic induction, but before rocuronium administration. On the arm with the TOF-Watch SX, a single train-of-four stimulation was provided (50 mA), followed by a 5-s, 50-Hz tetanic stimulation to decrease the stabilization period. After 1 min of train-of-four stimulation, calibration of the TOF-Watch SX was performed using the CAL 2 mode of the device to determine supramaximal stimulating intensity for each patient and adjust the T1 (single twitch) response to 100%. Once a stable baseline was achieved—defined as less than 5% variation in acceleromyography train-of-four ratios for a minimum of 2 min—rocuronium 0.6 mg/kg was administered. The same procedure was performed simultaneously on the arm with the TOFscan with the exception of the calibration of the device (set current intensity of 50 mA for all patients). The TOFscan and the TOF-Watch SX were then activated at the same time and train-of-four ratios measured every 15 s during onset of neuromuscular blockade and recovery of neuromuscular function.

Baseline train-of-four ratios were recorded before the administration of rocuronium. After rocuronium was administered, the time to achieve a train-of-four count of 0 was noted for both devices. During neuromuscular recovery, when the TOF-Watch SX displayed a train-of-four of 0.2, the train-of-four ratio on the TOFscan was recorded. In addition, a normalized train-of-four value was calculated after baseline measurements were obtained. For example, if the baseline train-of-four ratio with TOF-Watch SX was 1.1, a train-of-four value with the TOFscan was recorded when the TOF-Watch SX displayed a train-of-four of 0.18 (0.2/1.1 = 0.18). Train-of-four ratios with the TOFscan were then determined when the TOF-Watch SX displayed values of 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, and 1.0 (and up to 1.2 for patients with higher baseline measures), and again when normalized values were achieved. The times from neuromuscular antagonism until extubation and postanesthesia care unit admission were noted.

Demographic data and type of surgical procedure were recorded from the electronic medical record. Total rocuronium doses, number of redoses, anesthesia time, crystalloid volume, and temperature at the end of the surgical procedure were also collected from the electronic medical record.

The sample size needed to assess the agreement between train-of-four ratios obtained from with the TOFscan and those obtained with the TOF-Watch SX during spontaneous recovery of neuromuscular function was estimated based on a paired t test comparing the devices (StatsDirect Statistical Software Version 3.1.12 [November 6, 2017]; StatsDirect Ltd., United Kingdom), with a power of 90% to detect a difference between devices of at least 5% at a train-of-four ratio of 0.7 (primary endpoint) with a significance level of 5%. Assuming the SD of the train-of-four values to be 5%, and the correlation between the devices to be 0.3, 16 patients would be required. In order to account for patients who would not spontaneously recover to a train-of-four ratio of at least 0.9 by the end of the surgical procedure, 25 were enrolled. Good agreement between the TOFscan and TOF-Watch SX was considered to be a bias of less than 0.05 for the normalized (TOFscan minus TOF-Watch SX) values throughout the range of train-of-four ratio measurements.

Our initial primary data analysis plan was to determine the bias with limits of agreement (95% CI) for both nonnormalized and normalized train-of-four ratios measured with the TOF-Watch SX, and with the TOFscan at nine levels of block between 0.2 and 1.0, with one measurement per patient at each level of block, as reported by Claudius et al.¹²  This was planned to avoid the errors associated with doing a single analysis using possibly unequal numbers of paired measurements, which changed over time, from multiple subjects.¹³  An extension of the Bland and Altman method, based on ANOVA, has been reported by Olofsen et al. since the time the current study was designed.¹³  This improved method enables assessment of the agreement between methods over a range of repeated measurements, with unbalanced data, which they refer to as the modified true value varies method. This method was used to determine bias with limits of agreement (95% CI) for train-of-four measurements made by the two instruments using software made freely available by the authors (https://sec.lumc.nl/method_agreement_analysis/; accessed December 6, 2017). Limits of agreement were determined for levels of block between 0.2 to 1.0, with one pair of measurements per patient at each 0.1 level of block displayed by the TOF-Watch SX from 0.2 and 1.0 (i.e., up to nine measurements per patient). This analysis includes not only modified Bland–Altman plots, but also an assessment of the modified true value varies analysis. The latter includes Spearman rank correlation coefficient and intraclass correlation.

Patient characteristics, perioperative characteristics, and secondary outcomes (e.g., the times to achieve a train-of-four count of 0) are reported as the number of patients (%), median (interquartile range), or mean ± SD. Secondary outcomes were compared using the paired t test (StatsDirect). Mean differences and their 95% CIs were determined. The criterion for rejection of the null hypothesis was a two-tailed P < 0.05 for all comparisons.

Twenty-five patients were enrolled in the investigation. One patient was excluded when succinylcholine was used for neuromuscular blockade, and a second was excluded when the surgical procedure required the arms to be tucked to the side of the surgical bed. Complete data were collected on the remaining 23 patients.

The mean age of enrolled patients was 59 yr, and the mean weight was 78 kg. The most commonly performed procedures were urologic (43.5%) and gynecologic (30.4%). The median dose of rocuronium administered was 50 mg (35 to 50 mg), with a median number of redoses of 0 (0 to 1). The median duration of surgery was 184 min (144 to 203 min), and temperature at the end of the procedure was 36.3 ± 0.5°C.

Mean baseline train-of-four ratios were significantly higher in the TOF-Watch SX group (1.10 ± 0.05) compared to the TOFscan group (1.00 ± 0.01, P < 0.0001). Time to onset of complete neuromuscular blockade (train-of-four count of 0) was shorter using the TOF-Watch SX (128 ± 44 s) compared to the TOFscan (193 ± 64 s, P < 0.001; table 1).

Bias with limits of agreement between nonnormalized, as well as normalized, TOF-Watch SX measurements and TOFscan measurements during recovery from neuromuscular blockade are presented in table 2. Assessments of the modified true value varies analyses are presented in table 3.

The bias between the nonnormalized TOF-Watch SX measurements and the TOFscan measurements over the range of assessments was estimated to be 0.021 (95% CI, 0.001 to 0.040; table 2; fig. 3). The 95% limits of agreements were −0.100 to 0.141 (95% CI, −0.135 to −0.076 and 0.117 to 0.177, respectively). Diagnostics plots were inspected to ensure that the assumptions underlying the analysis were not violated. Spearman rank correlation coefficient, ρ, for the correlation between the size of the bias and the mean values was −0.002 (table 3), suggesting there is no trend in the bias over the range of values measured. The intraclass correlation (i.e., the ratio of between-subject variance and total variance), τ, was 0.490, which led the program to automatically choose to use the modified true value varies method for this analysis.

The bias between the normalized TOF-Watch SX measurements and the TOFscan measurements over the range of assessments was estimated to be 0.015 (95% CI, −0.002 to 0.031; table 2; fig. 4). The 95% limits of agreements were −0.097 to 0.126 (95% CI, −0.124 to −0.078 and 0.107 to 0.153, respectively). Diagnostics plots for this analysis were also inspected to ensure that the assumptions underlying the analysis were not violated. Spearman rank correlation coefficient, ρ, was −0.219, suggesting there is a slight negative trend in the bias over the range of values measured (table 3). The intraclass correlation, τ, was 0.353, which also led the program to automatically choose to use the true modified value varies method for this analysis.

The train-of-four ratios at reversal of neuromuscular blockade were significantly higher in the TOF-Watch SX group (0.94 ± 0.06) compared to the TOFscan group (0.92 ± 0.07, P = 0.006; table 1). The train-of-four ratios at extubation were also significantly higher in the TOF-Watch SX group (1.02 ± 0.06) compared to the TOFscan group (0.97 ± 0.04, P < 0.001).

This observational clinical trial, which assessed the agreement between train-of-four ratios obtained with the TOFscan and those obtained with the TOF-Watch SX during spontaneous recovery of neuromuscular function, found minimal bias and relatively narrow 95% limits of agreement. Because bias was less and the 95% limits of agreement slightly smaller when normalized TOF-Watch SX train-of-four values were compared with train-of-four values measured with the TOFscan, normalization of TOF-Watch SX train-of-four values improved agreement between the monitors when assessing neuromuscular recovery. As the TOF-Watch SX is no longer commercially produced, the findings observed in the current study suggest that the TOFscan is a suitable quantitative monitor for research studies and clinical care.

Acceleromyography was introduced into clinical practice in 1988 as a user-friendly monitor of neuromuscular function for the operating room. Acceleromyography is related to force of contraction by the formula force = mass × acceleration (Newton's Second Law of Motion). If mass is constant, acceleration is directly proportional to force. Compared to other quantitative devices, acceleromyography monitors require less stringent preparations, are easy to apply, and are small and portable. An important disadvantage of first-generation acceleromyography monitors, such as the TOF-Watch SX, is that acceleration of a muscle after nerve stimulation is measured in only one direction (perpendicular to the face of the transducer). However, stimulation of the ulnar nerve results in isotonic contractions of the adductor pollicis that are often in three dimensions, involving three joints, frictional forces, and deformation of tissues.¹⁴  The complex nature of the movement of the thumb after nerve stimulation may account for the lack of precision and accuracy reported in some studies with first-generation acceleromyography devices.

The TOFscan is a new acceleromyography monitor that employs a three-dimensional transducer to measure contraction of the adductor pollicis in all three planes. The transducer is encased in a thumb splint that is designed for optimal positioning and applies a preload. According to the manufacturer, no initial calibration of the device is required (although a CAL mode does allow calibration to be performed). At this time, only one other study has evaluated the ability of TOFscan to quantify neuromuscular function during onset and recovery of neuromuscular blockade.¹⁵  In that clinical trial, poor agreement was observed between the TOFscan and the TOF-Watch SX monitors during onset and early recovery of neuromuscular blockade (i.e., deep relaxation). However, at more complete levels of recovery (i.e., train-of-four ratio of 0.9), good agreement between the devices was observed. This investigation also found differences between the monitors during onset of neuromuscular blockade, with the time to a train-of-four count of 0 of nearly a minute longer for the TOFscan than for the TOF-Watch SX. However, in contrast to the study by Colegrave et al., our analysis found minimal bias between the two monitors during neuromuscular recovery. Differences in application of neuromuscular monitoring practices (baseline measurements and signal stabilization were not obtained, normalization was not performed, and preload was not applied in the study by Colegrave et al.¹⁵ ) likely accounted for the variability in findings between the investigations. In addition, data were acquired after reversal with neostigmine or sugammadex (rapid recovery) in the trial by Colegrave et al.,¹⁵  while data were collected during slower spontaneous recovery in the current investigation.

According to the Good Clinical Research Practice Guidelines, mechanomyography should be used as the comparator when new neuromuscular monitoring techniques are evaluated.¹¹  However, mechanomyography monitors are no longer commercially manufactured and are not available for contemporary research studies. The TOF-Watch SX is now used in the majority of research studies that require assessment of neuromuscular function and is currently considered the standard for objective neuromuscular monitoring. Several studies have demonstrated a good level of accuracy between acceleromyography and mechanomyography technologies,^16–18  although others have reported that acceleromyography and mechanomyography recordings of neuromuscular transmission cannot be used interchangeably.^19–21  Performance of acceleromyography, in relation to mechanomyography, can be improved with normalization of train-of-four values,^20,22  and the application of a preload.^12,23  Claudius et al. reported that when compared to mechanomyography measurements, preload increased the precision of acceleromyography measurements and normalization of train-of-four ratios decreased the bias of acceleromyography.¹²  If both acceleromyography and mechanomyography values were normalized, there was no significant bias between the two technologies.¹²  In theory, therefore, the TOF-Watch SX can be used to evaluate the performance of new quantitative monitoring technologies. In the current investigation, train-of-four values were measured with the TOFscan during neuromuscular recovery when the TOF-Watch SX achieved both actual and normalized train-of-four ratio endpoints, with normalized values likely reflecting better correlation with mechanomyography. Our clinical trial found that normalization of the TOF-Watch SX train-of-four ratios resulted in less bias and slightly smaller 95% limits of agreement between the monitors during neuromuscular recovery.

Baseline measurements of train-of-four ratios obtained before the administration of muscle relaxants often exceed 1.0 with the TOF-Watch SX.^24,25  The mechanism explaining this phenomenon has not been elucidated, but it may be due to the thumb not returning to the same baseline position after a train-of-four stimulus.²⁵  Therefore, in order to accurately assess complete recovery of neuromuscular function, train-of-four ratios measured with acceleromyography must be corrected or normalized by baseline values. For example, if a train-of-four ratio of 1.25 is observed before muscle relaxant administration, a train-of-four value of 0.9 represents incomplete neuromuscular recovery (0.9/1.25 = 0.72). In the current investigation, baseline mean ± SD TOF-Watch SX train-of-four ratios of 1.10 ± 0.05 were observed, while baseline TOFscan train-of-four ratios were 1.0 ± 0.01. Therefore, normalization of train-of-four ratios during neuromuscular recovery is not required with the TOFscan.

Time to onset of a train-of-four count of 0 was significantly longer with the TOFscan (mean, 193 s) than the TOF-Watch SX (mean, 128 s). Colegrave et al. also observed a poor correlation between the monitors until the time complete neuromuscular blockade was achieved.¹⁵  The authors reported a mean difference of 20% in times to obtain a train-of-four count of 0, with a difference of up to 3 min noted with one patient. Therefore, the use of the TOFscan may result in a delay in the performance of tracheal intubation. Based on these data, the TOFscan should not be used as a guide to determine the onset of maximal neuromuscular block. The reasons for this finding are uncertain, but it may be due to differences in stimulation intensity, preload applied to the thumb, or technology (e.g., gain, filtering).¹⁵  Conversely, it is possible that the TOFscan accurately reflects time until full neuromuscular blockade and that clinicians may be performing intubation too early based on TOF-Watch SX data.

The TOF-Watch SX was calibrated before the administration of rocuronium. Calibration adjusts stimulation current to determine supramaximal stimulation in the patient. Supramaximal stimulation is the electrical stimulus 15 to 20% above that necessary to produce contraction of all of the muscle fibers supplied to a nerve. Calibration increases the probability that the train-of-four responses will be within the measurement window and reduces the risk of significant background noise.¹¹  The TOFscan, in contrast, uses a fixed, noncalibrated current intensity, with a default output of 50 mA (although higher currents can be delivered in the manual or calibration modes). Helbo-Hansen et al. observed that repeatability and agreement between train-of-four ratios determinations were acceptable at a stimulating current of 50 mA.²⁶  Furthermore, Brull et al. noted no significant differences in train-of-four ratios when examining stimulating currents of 20, 30, and 50 mA.²⁷  In contrast, Schreiber et al. examined the agreement between calibrated and uncalibrated measurements performed at the end of surgery in 96 patients.²⁸  Discordance in train-of-four values was observed in 88 of the patients. We used a fixed stimulating current of 50 mA for the TOFscan, which was likely sufficient for the patients in the study, as obese patients (who may require higher stimulating currents)²⁹  were not enrolled.

There are limitations to this clinical trial. First, the application of the two devices was not randomized to the dominant and nondominant arms. However, a clinical trial by Claudius et al. concluded that significant differences in neuromuscular responses were not observed between the two arms.³⁰  Second, raw control train-of-four values measured with the TOFscan in the investigation were 1.0 (in contrast to 1.10 with the TOF-Watch SX). There are a few explanations for why the train-of-four values were closer to unity with the TOFscan. According to the manufacturer, the thumb splint was designed (shape and the hardness of the splint) to minimize the risk of control train-of-four values exceeding 1.0 (personal written communication, December 2017, Thierry Bagnol, IDMED, Marseille, France). It is also possible that the new three-dimensional transducer technology limits raw values greater than 1.0 by more accurately assessing thumb position at rest and with movement. In addition, as with the TOF-Watch and TOF-Watch SX, the displayed train-of-four ratio is limited to 1.0 (although the train-of-four bar graph presents the absolute amplitude response). If the second twitch (T2) is greater than the first twitch (T1), then the train-of-four value shown is calculated using the fourth twitch/second twitch (T4/T2) ratio, not the fourth twitch/first twitch (T4/T1) ratio. It is interesting to note that 25% of the displayed baseline values exceeded 1.0 with the TOFscan. Since most clinicians would be unlikely to use the REF mode of the TOFscan to measure raw baseline train-of-four values and perform normalization, the current study was conducted to reflect standard practices. Third, the stimulating current intensity delivered by the TOF-Watch SX after calibration was not recorded; therefore, it is uncertain whether a stimulation intensity of more than 50 mA was used. Finally, studies have demonstrated that electromyography has the closest correlation with mechanomyography during onset and recovery of neuromuscular blockade, and investigators have stated that electromyography should be considered the standard for the objective assessment of neuromuscular function and for the evaluation of new quantitative monitoring technology.^7,31  At this time, however, no stand-alone electromyography devices are available for clinical or research applications.

In conclusion, this observational study demonstrated good agreement between the TOF-Watch SX with calibration and preload application and the uncalibrated TOFscan during neuromuscular recovery. Normalization of the TOF-Watch SX train-of-four values decreased the bias and slightly decreased the 95% limits of agreement. However, the time to determine onset of complete neuromuscular blockade was significantly longer with the TOFscan. Our findings showed that the TOFscan may be used in clinical practice to establish the presence or absence of residual muscle weakness before tracheal extubation.

Supported by the Department of Anesthesiology, NorthShore University HealthSystem (Evanston, Illinois).

Dr. Murphy has served on the Advisory Board and as a speaker for Merck (Kenilworth, New Jersey). Dr. Greenberg has served as a speaker for CASMED (Branford, Connecticut). The remaining authors declare no competing interests.