Accuracy of Determining Hemoglobin Level Using Occlusion Spectroscopy in Patients with Severe Gastrointestinal Bleeding

• Whether new, noninvasive hemoglobin measurements are as reliable as traditional, capillary-based methods in the setting of acute hemorrhage has not been critically examined

• In a prospective equivalence study in intensive care unit patients admitted for gastrointestinal hemorrhage, a noninvasive occlusion spectroscopy method lacked accuracy compared with a traditional, capillary-based method to measure hemoglobin, using the laboratory cooximeter as a criterion standard

• The inaccuracy of the noninvasive method was influenced by neither infusion of vasopressor agents nor site of measurement

PATIENTS suffering from acute gastrointestinal bleeding frequently require a blood transfusion.1–3These patients are usually admitted to an intensive care unit (ICU) for urgent diagnosis and when prompt therapeutic decisions are required. Hemoglobin level is regularly determined to provide further information about a persistent or recurrent bleeding incident and to guide blood transfusion decisions.4 

Although considered as the reference method, determination of hemoglobin using a laboratory cooximeter requires withdrawal of blood and a noncompressible delay until a result is obtained. The availability of a more rapid, precise, and reproducible method of determining hemoglobin concentration would be particularly helpful. The capillary-based method, which allows total hemoglobin determination at the bedside from a single drop of blood, is probably the method most used by physicians. However, its accuracy in clinical trials remains controversial, notably in patients with severe hemorrhage.5–10 

Recently, a continuous and totally noninvasive technology to measure hemoglobin concentration, based on pulse cooximetry, has been proposed. This method discerns the distinctive light-absorption characteristics of the different hemoglobin species and applies proprietary algorithms to determine total hemoglobin levels.11,12This device would reduce the time needed to obtain results and, by its continuous nature, alert physicians to additional occult blood loss. Although this device was initially validated in healthy volunteers undergoing normovolemic hemodilution,13its use in clinical conditions has revealed serious limitations.14–20Its lack of accuracy may be because of the major changes in microvascular tone that are frequently associated with acute hemorrhage,21and because of the use of vasopressor agents given to restore perfusion pressure.22 

A new device, the NBM-200MP, has been developed by OrSense^®(Nes Ziona, Israel). This combines low-perfusion oximetry and occlusion spectroscopy, and this innovative and highly sensitive technology should allow us to monitor the oxygen saturation of hemoglobin with improved accuracy, in particular, in patients with poor microperfusion.23The recent development of this new algorithm may be very useful in assessing hemoglobin level in critically ill patients.

The authors tested the hypothesis that OrSense^®’s occlusion spectroscopy could be considered equivalent to the capillary-based method in determining hemoglobin at the bedside in patients admitted to our ICU for gastrointestinal bleeding. By determining hemoglobin level using a laboratory cooximeter as the reference method, we have conducted an equivalence trial in which the accuracy of the occlusion spectroscopy and capillary blood methods could be compared.

Approval was granted by our institutional review board (Comité de Protection des Personnes Sud-Ouest et Outre Mer III, Bordeaux, France). The agreement number for this study was DC2011/50. Because data were collected while the care of patients conformed to the standard procedures currently used in our institute, authorization was granted to waive informed consent for this study.

We conducted an observational, prospective, and equivalence study during a 5-month study period between July and November 2011. All patients admitted to our 22-bed mixed medicosurgical ICU at the Haut-Lévêque Hospital (CHU de Bordeaux) for severe gastrointestinal bleeding were enrolled prospectively. To preferentially include patients suffering from an acute hemorrhage, those in whom the last blood exteriorization occurred at more than 12 h previously were excluded. Patients suffering from septic shock, patients with a skin or fingernail disease, patients with hand or finger edema, and moribund patients with a life expectancy of less than 24 h were also excluded. Monitoring of patients included continuous electrocardiography, intermittent noninvasive measurement of arterial blood pressure, pulse oximetry, and urinary output. Sedation, mechanical ventilation, and administration of catecholamine and antibiotics were left to the discretion of the attending physician. Similarly, more invasive techniques, such as central venous and arterial or urinary catheters, were determined according to overall clinical management.

In patients admitted to our ICU for gastrointestinal bleeding, biologic tests at admission systematically included a coagulation test and a venous blood cell count using venous hemoglobin (vHb) measurements obtained with a laboratory cooximeter (LH 780-automate; Beckman Coulter, Brea, CA). The precision of this device, as reported by the manufacturer, is less than 0.8%.**The hemoglobin level of capillary blood (cHb) was determined simultaneously at the bedside by taking a single drop of blood from a fingertip skin puncture, which was then measured using a portable HemoCue^®Hb 201⁺(HemoCue^®, Inc., Meaux, France) hemoglobin meter for immediate assessment. In accordance with our practice, three successive measurements of cHb were performed from three different drops of blood extracted from one fingertip puncture. The cHb level was defined as the mean of these three measurements. Because the manufacturer recommends the use of a single drop to fill the cuvette, with the most representative being the third or fourth drop forming at the puncture site,24a post hoc  sensitivity analysis compared the accuracy of the capillary-blood method using only the third drop and the mean of three drops. The vHb level was always taken from a new venipuncture site, and an effort was made to take the vHb and cHb blood samples from an extremity that did not have intravenous fluids being actively infused.

The NBM-200MP was set up at admission of the patient. As recommended by the manufacturer, the ring-shaped sensor was fitted onto the patient’s thumb, which was temporarily and gently squeezed to oversystolic pressure, similar to that which occurs during blood pressure measurements. A highly sensitive optical system, using an array of calibrated light sources, measured light absorption and scattering. A desktop monitor then analyzed the blood constituents and displayed the OrSense^®hemoglobin result (SoHb) after approximately 2 min. A sensitivity post hoc  analysis that evaluated the accuracy of the hemoglobin measurement at the forefinger site (So^fHb) also assessed possible variability between the sites. Consequently, the hemoglobin measurement at the thumb (So^tHb) and So^fHb were successively recorded while blood samples were being drawn to determine vHb and cHb.

When hemodynamic monitoring included a radial arterial catheter, the ring sensor was preferentially fitted onto the fingers from the opposite hand. Conforming to the standard procedures for patients admitted into our ICU for severe gastrointestinal bleeding, measurement of the hemoglobin level was repeated every 8 h and included simultaneous determination of venous, capillary, and occlusion spectroscopic hemoglobin level. We arbitrarily defined the first 24 h after admission to the ICU as the study period. Consequently, the length of the study included four hemoglobin determinations. As mentioned above, this study conformed to the standard care given to patients in our ICU except for the use of the OrSense^®sensor.

The primary endpoint was the percentage of inaccurate absolute measurements that were considered clinically unacceptable and defined as a greater than 15% difference compared with the laboratory reference method or their unavailability for any technical reason.18Although the relative difference when assessing the accuracy of the device appears to be scientifically relevant, it is less simple for the clinician to calculate a percentage deviation than an absolute difference. Consequently, the post hoc  sensitivity analysis considered an inaccuracy threshold of an absolute difference of ±1 g/dl.

The secondary endpoints included bias, precision, and limits of agreement for the punctual measurement. As the OrSense^®device uses a technology that may be sensitive to the state of the microcirculation, the precision of SoHb was also compared between patients receiving or not receiving vasopressor agents such as norepinephrine, epinephrine, vasopressin, somatostatin, or octreotide. Finally, we determined the number of erroneous decisions made regarding transfusions (both too many and too few) that would have been made if a given threshold of hemoglobin concentration (8, 9, and 10 g/dl) had been used to prescribe a red blood cell transfusion.

The required sample size was calculated from the proportion of inaccurate absolute measurements (see above for definition). Considering one study recently published by our group18and other previous trials,6,13a proportion of 20% of inaccurate measurements was estimated. Assuming an α level of 0.05 and a β risk of 0.10, a proportion of inaccurate measurements in the HemoCue^®group of 20%, and a prespecified equivalence limit of 30% of inaccurate measurements in the SoHb group, we calculated that 272 measurements were required to test the equivalence between HemoCue^®and SoHb (nQuery Advisor 6.0; Statistical Solutions Ltd., Cork, Ireland). Because four consecutive measurements should be performed in each patient, a total of 68 patients were required. We preplanned to perform an interim analysis when half of the total number of patients had been enrolled (n = 34).

The present observational study compared the accuracy of two methods to determine hemoglobin level noninvasively—the occlusion spectroscopic method and the capillary blood method—by assessing inaccuracy of the measurement, defined as a greater than 15% difference compared with the reference laboratory method. The linear relationship between paired measurements was performed using a simple regression model after correcting the data using Poon’s adjustment, which took into account the repeated nature of the measurements.25Concordance between the two measurements was evaluated using the Bland–Altman method, with a correction for repeated measures.26–28Bias was defined as the mean difference between the test and reference measurement, whereas precision was defined as the mean absolute value of this difference. The limits of agreement were calculated as the bias ± 1.96 SD. We compared measurements obtained between patients who received or did not receive norepinephrine as a post hoc  analysis.

Data are expressed as mean ± SD for normally distributed continuous variables, medians and interquartile ranges for nonnormally distributed continuous variables, and numbers (percentage of patients) for categorical data. The normal distribution of continuous variables was assessed using skewness and kurtosis statistical tests. Comparisons between means and proportions were performed using Student t  test and the chi-square test, respectively. All P  values were two-tailed, and a value of P < 0.05 was considered significant. Statistical analysis was performed using NCSS 2007 software (Statistical Solutions) and SAS software version 9.3 (SAS Institute, Inc., Cary, NC).

As interim analysis of the first 34 consecutive patients demonstrated that SoHb could not be considered equivalent to cHb, the trial was terminated prematurely. The results described below are for these 34 consecutive patients admitted to our ICU for severe gastrointestinal bleeding during the 5-month study period (fig. 1). The characteristics of the patients are summarized in table 1. Twenty-three patients (68%) received four or more transfused red blood cell concentrates. Ten of the patients in our study also received vasopressor agents: seven received norepinephrine at a median maximal dose of 0.41 µg·kg^–1·min^–1(interquartile range, 0.06–0.55 µg·kg^–1·min^–1) and three received octreotide, a synthetic analog of somatostatin at a fixed dose of 600 µg for 24 h. Any patient included in this study received epinephrine and/or vasopressin.

As four consecutive measurements should be performed, a total of 136 paired measurements should be theoretically available. For one patient, only three measurements of vHb could be realized. Consequently, a total of 135 vHb measurements were performed, with a mean hemoglobin concentration of 9.9 ± 1.8 g/dl. From the 135 theoretical simultaneous measurements of So^fHb and So^tHb, three (2.2%) and two (1.4%) of these were, respectively, unavailable because the sensor was unable to detect sufficient blood perfusion in the patient’s fingertip. Measurements of cHb levels were available at all times and provided appropriate data without any device malfunctions. The accuracy of the capillary-based method was similar when either only the third drop or the mean of three drops was used to determine hemoglobin level (table 2).

The proportion of outliers (and thus inaccurate measurements) when hemoglobin level was determined by the capillary method was 24% (95% CI, 20–28%). For So^tHb and So^fHb, the proportions of inaccurate measurements were, respectively, 47% (95% CI, 43–51%) and 53% (95% CI, 49–57%), which meant that we could reject the hypothesis of equivalence between SoHb and cHb regardless of the site of measurement. Considering an inaccuracy threshold of ±1 g/dl, the proportion of inaccurate measurements for cHb and So^tHb were, respectively, 37% (95% CI, 30–44%) and 62% (95% CI, 57–67%).

Although statistically significant, the relationship between So^tHb and vHb was poor (r  ²= 0.50, P < 0.001) (fig. 2A). A similar result was obtained for the So^fHb thumb measurement (r  ²= 0.30, P < 0.001) (fig. 2B). Conversely, cHb was well correlated with vHb (r  ²= 0.78, P < 0.001) (fig. 2C). When vHb was compared with So^tHb and cHb, the mean biases were, respectively, −0.4 ± 2.0 and 0.8 ± 1.2 g/dl (P < 0.05) (fig. 3). All parameters of the analysis of concordance are summarized in table 3.

As four measurements should have been theoretically performed on each patient, a total of 204 changes in hemoglobin levels could have been analyzed. In actuality, for So^tHb and cHb, respectively, 195 (96%) and 201 (98%) changes were available. The relative changes in vHb were significantly correlated with the changes in both So^tHb (r  ²= 0.20, P < 0.01) and cHb (r  ²= 0.56, P < 0.001) (fig. 4, Aand B, respectively).

Bias, precision, and the proportion of inaccurate measurements were compared between patients receiving (n = 10) and not receiving vasopressor agents (n = 24). We obtained, respectively, 36 and 97 paired measurements from these patients. The accuracy of occlusion spectroscopy to determine hemoglobin level in patients receiving and not receiving vasopressor agents is summarized in table 4. Although the inaccuracy in determining hemoglobin level by So^tHb tended to be increased in patients receiving vasopressor agents, there was no significant difference between the groups (table 4).

The proportion of failed transfusions, if a decision had been based on a noninvasive method, would have been larger when using occlusion spectroscopy with a threshold of 9 and 10 g/dl, but not for 8 g/dl (table 5). However, the total number of erroneous decisions, including excessive and failed transfusions, would have been comparable between both methods regardless of the threshold of transfusion.

This equivalence trial was conducted on critically ill patients suffering from severe gastrointestinal bleeding in the ICU. The main findings were that (1) the accuracy of the pulse OrSense^®occlusion spectroscopy method of determining hemoglobin was not equivalent to the capillary-based method, (2) the lack of accuracy of SoHb persisted regardless of the site of measurement (thumb or forefinger), (3) determination of the hemoglobin value by using the OrSense^®device might have been responsible for the estimated high proportion of failed transfusions if SoHb had been used to guide red blood cell transfusions, and (4) the accuracy tended to decrease in patients receiving norepinephrine.

Many experts consider that the decision to prescribe red blood cell transfusions should be based on the physiologic state or on the estimated amount of blood lost rather than on hemoglobin concentration.29However, estimating blood loss can be complex and is frequently overestimated, especially when rapid fluid resuscitation is given. Overall, in medical practice, most clinicians take hemoglobin level into account when making treatment and transfusion decisions, in particular, during gastrointestinal hemorrhage, as bleeding is often not exteriorized immediately, and often only during a second bleed.

To determine hemoglobin level, the use of a laboratory cooximeter is the reference method. However, the time needed to obtain results can be lengthy, and such a delay can be prejudicial during the urgent management of patients suffering from severe hemorrhage. Portable cooximeters have been developed to obtain rapid measurements of hemoglobin from a single drop of capillary blood (from a fingertip skin puncture), but this technique is somewhat invasive and its accuracy remains controversial.5–10 

A new technology, the Signal Extraction Technology (Masimo Corp., Irvine, CA), has been developed to improve the accuracy of oxygen saturation measured by pulse oximetry values, and was designed to noninvasively measure hemoglobin concentration using multiple wavelengths of light.30,31Although some studies have evaluated the accuracy of pulse cooximetry as acceptable to determine hemoglobin concentration,32–34several other studies, conducted under clinical conditions and particularly during severe hemorrhage, have found this method to have less than optimal accuracy.14–18Furthermore, we recently evaluated Signal Extraction Technology in severe gastrointestinal bleeding and reported that its accuracy was not clinically acceptable, notably because of dysperfusion observed in patients with the most severe bleeding.18That is why the innovative NBM-200MP technology, developed by OrSense^®, based on the association of low-perfusion oximetry combined with occlusion spectroscopy, could be very useful.

We decided to arbitrarily limit the length of the study period per patient to the first four measurements of hemoglobin level to minimize bias and to limit the study time to the acute management period. Patients were critically ill and suffered from severe hemorrhage, with most requiring a blood transfusion during the 24-h study period. Moreover, we chose to enroll patients with acute gastrointestinal bleeding regardless of their hemodynamic status or severity of hemorrhage, and 10 patients required vasopressor agents. Because vasopressors may alter vasomotor tone, we tested the hypothesis that the performance of the OrSense^®technology to measure hemoglobin level may be worse for these patients. We observed that its accuracy tended to be decreased even though the doses of vasopressor agents were relatively low.

When major bleeding occurs, blood loss is also responsible for vasoconstriction. This could be why the accuracy of SoHb was so poor under these circumstances. In our study, this occurred not only when the hemodynamics were precarious but also after hemodynamic restoration using volume expansion and blood transfusion. The need for norepinephrine perfusion exacerbated this.

The primary endpoint of this study was the proportion of inaccurate measurements, which was defined as the relative difference. In our study, the proportion of inaccurate measurements also included the proportion of unavailable measurements that must be considered by clinicians as unacceptable values. Based on this definition, we found 24% of measurements to be inaccurate for the capillary-based method. Although most authors have defined inaccurate measurement as an absolute difference (i.e. , >1 g/dl) and have excluded patients from their analysis in whom measurement of hemoglobin level by a device was technically unavailable,13,17the proportion of inaccurate measurements that we found appears to be consistent with previous studies.6,13,14,18 

For the occlusion spectroscopy–based method, the proportion of inaccurate measurements appears to be clinically unacceptable and is globally comparable to the oximetry-based method.14,18Despite significant correlations with the reference method for both SoHb and cHb, the Bland–Altman method suggests that the bias, precision, and limits of agreement for SoHb cannot substitute for the reference method. These findings were similar regardless of the measurement site: the thumb or the forefinger. In contrast, cHb appeared to predict vHb more reliably in our study.

Previous studies have evaluated the accuracy of portable hemoglobin meters to determine total hemoglobin in capillary blood. Although some authors recommend their use to guide transfusion in clinical practice,7,9their accuracy remains controversial. In patients admitted to ICUs for gastrointestinal bleeding, Van de Louw et al. reported a low mean difference (bias, 0.06 ± 0.87 g/dl) between hemoglobin level determined by a laboratory and that of capillary blood samples.6These authors found that 21% of the differences were greater than 1 g/dl. Rippmann et al. observed a bias of 0.6 ± 0.6 g/dl between hemoglobin levels determined by a central laboratory and those from a HemoCue^®system for capillary blood.7A similar result was recently reported in patients admitted to a surgical ICU.5These findings are consistent with our present study, as we found a mean bias of 0.8 ± 1.2 g/dl, and the proportion of estimated accurate measurements was 76%.

The accuracy of the NBM-200MP had not been reported in the literature until recently: Gayat et al. reported the first clinical study that assessed its accuracy.35In an emergency department, they enrolled 300 patients who needed hemoglobin measurement for any reason. Despite a relatively limited bias, they found large limits of agreement (−3.01 to +3.42 g/dl in their population. These results are comparable to those reported by our study. Moreover, Gayat et al. enrolled less severe patients than we did, as demonstrated by only 6% of patients having an active or recent history of bleeding and no report of any vasopressor use. Thus, better results should have been expected in their population. In our study, the performance of the NBM-200MP worsened in severe patients with acute bleeding, blood loss, and fluid shifts. However, the proportion of patients in whom the measurement of hemoglobin was technically impossible appeared to be lower than that observed using the oximetry method.17,18 

To give clinicians a greater sense of our findings and their impact on patient management, we determined the proportion of erroneous decisions regarding transfusion procedures that would have been made if the data from the devices had been taken into account. We arbitrarily fixed three transfusion thresholds at 8, 9, and 10 g/dl. The NBM-200MP induced failed transfusion decisions more frequently compared with the HemoCue^®, and this was probably related to overestimation of hemoglobin level.

The following points should be considered when assessing the clinical relevance of our results. First, this study enrolled patients suffering from gastrointestinal bleeding. As a consequence, the results have to be extrapolated cautiously, even for other acute hemorrhage syndromes. Second, although we monitored core temperature during the study and no hypothermia was reported, distal hand temperatures were not recorded. Thus, distal hypothermia cannot be totally ruled out, as hypothermia could induce vascular changes and potentially alter hemoglobin readings. Third, our study included only severe patients (i.e. , patients who needed ICU management). Thus, the accuracy of the NBM-200MP should be better in others suffering from less severe gastrointestinal bleeding. For them, blood loss may be less and vasomotor tone should be less altered, resulting in better precision. Fourth, the mean of three consecutive values of cHb were used for each measurement. Even if this is considered good clinical practice,36it could bias the analysis by artificially reducing the variance in cHb compared with the single SoHb measurement. Finally, a bias caused by dependent differences cannot be ruled out because changes in hemoglobin level were obtained by comparing all possible differences between four consecutive measurements.

In conclusion, the noninvasive determination of hemoglobin using occlusion spectroscopy lacks accuracy in critically ill patients presenting with severe gastrointestinal bleeding and cannot be considered equivalent to the capillary blood method. The inaccuracy was not influenced to any extent by the infusion of vasopressor agents. An overestimation of hemoglobin concentration by the device might tend to increase the number of failed transfusions.

The authors thank Erwan Floch, Pharm.D. (Newmed Publishing Services, Paris, France), for reviewing the manuscript.