Intrapatient Reproducibility of the BISxp® Monitor

We conducted a nonrandomized, prospective, comparative, observational study in patients undergoing elective surgery during general anesthesia. Approval was obtained from the institutional review board (Washington University School of Medicine, St. Louis, Missouri), and all patients recruited provided written informed consent. Fourteen patients were enrolled into the study. Two were excluded. One had surgery postponed, and the BISxp® did not provide readings for the other patient. All participants in the study had two BIS Quatro® electrode strips from BISxp® devices placed on each side of their foreheads for the whole duration of their anesthetic, as described in recent studies.10–12The manner of placement was the same in all the participants (fig. 1). In the last three patients enrolled, we applied three electrode strips. Two were applied as shown in figure 1, and the third was placed on the same side as, parallel to, and just above the left-sided electrode strip. The purpose of this third electrode was to check for intrapatient reproducibility for electrodes placed on the same side of the forehead. Freestanding BISxp® devices were connected to the electrodes. The impedances were measured for each set of electrodes to ensure optimal electrode contact defined as 7.5 kΩ or less as required by the manufacturer. The clocks on the BIS® devices were synchronized to ensure discrepancy of 1 s or less between the clocks, and BIS data—updated every 5 s—were downloaded for analysis to laptop computers. Anesthesia was administered according to the preference of the attending anesthesiologist. Patients were excluded if they had a history of stroke, dementia, organic brain disease, or carotid bruit or when ketamine—an agent known to induce unusual changes in electroencephalographic activity13—was used as part of the anesthetic technique. A structured questionnaire4was conducted in the postoperative recovery area and repeated on the first postoperative day in the hospital ward to identify possible episodes of explicit recall of intraoperative events.

The primary outcome measure of the study was the relation of BIS readings between two devices during the course of general anesthesia. Bland-Altman tests of agreement and bias14were used as well as regression coefficients between the BIS recordings. We also performed time-binned analysis where we included only time periods of 30 s (six sequential paired data points) or longer when there were BIS-defined differences in planes of anesthesia between the two BISxp® devices. We analyzed individual data points to establish how many and what percentage of the BIS readings differed by 10% or by 10 ABUs. We further performed time-binned analysis (for time periods ≥ 30 s) as described above to determine whether these differences were sustained or whether they were just sporadic. We did these analyses both for electrodes placed on opposite sides of the forehead (all 12 patients) and for electrodes placed on the same side of the forehead (the last 3 patients).

The patients’ characteristics are shown in table 1. All the patients were right-handed. None of the patients included in the study had explicit recall of intraoperative events. From the 12 patients, 22,860 concurrent paired BIS data points were recorded from the electrodes on opposite sides of the forehead. From the last 3 patients, 8,316 concurrent paired BIS data points were recorded from the electrodes on the same side of the forehead.

Overall, there was no bias (< 1) between the readings from the BIS® electrodes that were placed on opposite sides of the forehead, and the 95% limits of agreement between them were from −18 to + 17 (fig. 2). Correlation (R ) was 0.8, and regression (R  ²) was 0.65. Overall, 30% of concurrent BIS readings suggested conflicting depths of anesthesia, as defined in previous studies and by the manufacturer of the BIS® (table 2).12The readings from electrodes that were placed on the same side of the forehead had a bias of −2.8, and 95% limits of agreement between them were from −17 to +11. Thirty-two percent of these concurrent readings suggested conflicting depths of anesthesia. The correlation (R ) was 0.83, and the regression (R²) was 0.68.

With time-binned (six sequential paired data points) analysis, 10.7% of the time, the simultaneous readings of the BISxp® devices from electrodes on opposite sides of the forehead suggested sustained (≥ 30 s) differences in depth of anesthesia (table 3). From the electrodes that were placed on the same side of the forehead, 14% of the time, the simultaneous BIS readings suggested sustained (≥ 30 s) differences in depth of anesthesia (table 4). The results showing differences between BIS readings of 10% and 10 ABUs (including sustained differences for ≥ 30 s) are shown in tables 3 and 4. Six percent of the time, there were sustained periods of 30 s or greater where the BIS readings from electrodes on opposite sides of the forehead differed by 10 or more arbitrary BIS units. Figures 3 and 4show photographic examples of concurrent BIS trends in 2 of the patients. One of the photographs shows sustained differences between the BIS readings for a portion of the anesthetic period, and the other shows an example of good agreement between the readings for a portion of the anesthetic period. Figure 5shows the concurrent BIS trends over time for each of the 12 patients for the whole anesthetic period.

An ideal monitor of anesthetic depth should be affected similarly and predictably by different anesthetic drugs and should reproducibly reflect anesthetic depth in different patient populations. An ideal monitor should display good interpatient and intrapatient reliability.15,16The results of the current study suggest that the BISxp® did not consistently display intrapatient reproducibility in the patients that we studied.

If the BIS readings were consistently divergent, this could suggest technical error. However, for much of the time, the readings were concordant. In addition, the BIS reading was displayed as a solid number, signal quality index was always good, and the impedance was acceptable (≤ 7.5 kΩ). These are criteria by which the manufacturer recommends that the reliability of the BIS reading may be judged. We initially suspected that the differences between the devices could represent lateralization: The BIS may be different when obtained from right- and left-sided electrodes. However, the results from the patients that had electrodes placed on the same side also showed inconsistent intrapatient reproducibility. There was no discernable overall hemispheric bias, although all of our patients were right-handed. The BIS has been shown to be affected by organic brain disease.17,18It is unlikely that the inconsistent intrapatient reproducibility was attributable to unrecognized, unilateral brain pathology, because we excluded any patients with neurologic disease by careful preoperative history and physical examination. Moreover, with all the patients, there were sustained periods of discordance between the BISxp readings (table 3and fig. 5). It is improbable that all the patients had unrecognized unilateral brain pathology.

One of the patients, patient 5 (table 2), had results that were strikingly different from the rest of the patients. The BIS readings showed the greatest discordance, and there was bias between the BIS readings. One explanation for this discordance is that there was artifact or a technical error. However, as with all of the patients, there was no evidence of technical error. The signal quality index was consistently good for both BIS readings, yielding solid BIS numbers, and the impedance was acceptable. The discrepancies between traces could hypothetically have been reflective of occult unilateral brain pathology, which may be a potential source of discrepant hemispheric BIS readings. This explanation is not likely for this patient. A magnetic resonance imaging scan of the head revealed no organic brain injury. In addition, the discordance was not consistent or sustained; for part of the anesthetic period, the two BIS readings were reproducible for this patient (fig. 5). There are reportedly four components parameters that are combined to provide a composite BIS index.19Suppression is not weighed heavily in the BIS index until it exceeds 40%.19The differences between the BISxps® in the suppression ratio readings may have contributed to the simultaneous and sustained discordant traces. The extent of discordance in this patient is concerning.

Differences between monitoring devices may be statistically significant but clinically unimportant. Although 30% of the concurrent BIS readings suggested different depths of anesthesia as defined by the manufacturer, this does not necessarily mean that 30% of the time the two monitors displayed clinically relevant divergence. This could be reflective either of sporadic differences between the simultaneous BIS readings or of small differences between the readings. We therefore analyzed the data for fixed differences between the readings (> 10% and > 10 ABUs) and we determined the proportion of simultaneous BIS readings that differed uninterruptedly for at least 30 s. When only 30-s uninterrupted periods (six sequential BIS readings on each device) were included, 10.7% of the time, the BISxp® devices suggested different anesthetic planes from each other, which would suggest different anesthetic management. With the criteria of a difference between simultaneous BIS readings of at least 10 ABUs sustained for at least 30 s, the readings were concordant 94% of the time. This could be interpreted in two ways. On the one hand, the broad agreement between the BISxp® monitors 94% of the time may be perceived as encouraging. On the other hand, if an accurate BIS reading is considered important in guiding appropriate anesthesia, the 6% sustained (> 30 s) and meaningful (> 10 ABUs) difference between two concurrent BIS readings may be concerning. The graphs showing the concurrent BIS trends in each patient (fig. 5) show that the extent of concordance differed among patients and that for most patients, there were periods when the simultaneous BIS trends were discordant.

In the B-Aware study, one of the patients reportedly experienced explicit recall from a period during surgery when the BIS readings were in the range of 55–59 ABUs.4The authors explain, “This episode suggests that awareness can arise when the BIS is at the upper limit of the recommended (40–60) range.”4They suggest that a response to this might be to ensure that the BIS is less than 55 ABUs if awareness is to be avoided in most cases.4Another group of investigators has suggested that a persistently low BIS reading (≤ 45 ABUs) is associated with adverse long-term outcome.20If the findings of these studies are applied to routine anesthetic practice, providers should strive to maintain the BIS within a very narrow range of 10 (45–55) ABUs. The results of our study suggest that the BISxp® may lack the accuracy and reproducibility (tables 2–4) to be able to achieve this. If the target for the BIS number is 45–55 ABUs and we aim for the middle of that range (50 ABUs), a deviation of a mere 5 ABUs in either direction will take the reading out of the desired range. From our results, when two BISxp® are placed simultaneously, if one BIS reading is 50 ABUs, the other would read 45 ABUs or less or 55 ABUs or greater approximately 17% of the time (≥ 10% discordance sustained for ≥ 30 s).

There have been several recent studies that have suggested that using BIS® during general anesthesia may decrease the incidence of explicit recall.4,7Although there has been some criticism of these studies base on methodologic issues, it is conceivable that BIS monitoring reduces explicit recall despite the inconsistency of its intrapatient reproducibility. The presence of any monitor designed to alert the anesthesia provider to the problem of explicit recall may heighten vigilance and decrease the likelihood of explicit recall. Also, these monitors may be helpful in critical situations such as when no volatile anesthetic agent is being delivered or when an intravenous anesthetic infusion has run out. Although the BIS, a processed electroencephalograph, is only a surrogate marker of consciousness, it may be more reliable in this regard than other surrogates, such as volatile anesthetic concentration. Further research is needed to address this.

In summary, the results of this study suggest that the BISxp® does not consistently display intrapatient reproducibility. These results are at variance with the manufacturer’s claim that the BIS® monitor provides a reproducible and “reliable single number that represents each patient’s level of consciousness.” The results of this study reinforce the sentiment expressed on the Aspect Medical Web site: “Clinical judgment should always be used when interpreting the BIS in conjunction with other available clinical signs. Reliance on the BIS alone for intraoperative anesthetic management is not recommended.”

Silvio Breno, M.B.B.Ch., F.F.A.(SA) (Consultant Anesthetist, Department of Anesthesia, Area 361 Johannesburg Hospital, Johannesburg, South Africa), provided the inspiration for this study.