Vital parameter data collected in anesthesia information management systems are often used for clinical research. The validity of this type of research is dependent on the number of artifacts.
In this prospective observational cohort study, the incidence of artifacts in anesthesia information management system data was investigated in children undergoing anesthesia for noncardiac procedures. Secondary outcomes included the incidence of artifacts among deviating and nondeviating values, among the anesthesia phases, and among different anesthetic techniques.
We included 136 anesthetics representing 10,236 min of anesthesia time. The incidence of artifacts was 0.5% for heart rate (95% CI: 0.4 to 0.7%), 1.3% for oxygen saturation (1.1 to 1.5%), 7.5% for end-tidal carbon dioxide (6.9 to 8.0%), 5.0% for noninvasive blood pressure (4.0 to 6.0%), and 7.3% for invasive blood pressure (5.9 to 8.8%). The incidence of artifacts among deviating values was 3.1% for heart rate (2.1 to 4.4%), 10.8% for oxygen saturation (7.6 to 14.8%), 14.1% for end-tidal carbon dioxide (13.0 to 15.2%), 14.4% for noninvasive blood pressure (10.3 to 19.4%), and 38.4% for invasive blood pressure (30.3 to 47.1%).
Not all values in anesthesia information management systems are valid. The incidence of artifacts stored in the present pediatric anesthesia practice was low for heart rate and oxygen saturation, whereas noninvasive and invasive blood pressure and end-tidal carbon dioxide had higher artifact incidences. Deviating values are more often artifacts than values in a normal range, and artifacts are associated with the phase of anesthesia and anesthetic technique. Development of (automatic) data validation systems or solutions to deal with artifacts in data is warranted.
Anesthesia information management systems are increasingly used for anesthesia recordkeeping and can provide useful data for patient care, research, or medicolegal purposes
It is recognized that electronically stored vital parameter data may include artifacts
Different systems have different ways to identify artifacts
There are some data to suggest the amount of artifacts may be greater in pediatric anesthesia
In the particular anesthesia information management system used by the authors, the amount of artifacts was low for heart rate and oxygen saturation and higher for noninvasive and invasive blood pressure and end-tidal carbon dioxide
Values outside the normal range have a higher amount of artifacts than values within the normal range
The amount of artifacts varies with anesthetic technique and phase of anesthesia
ANESTHESIA information management systems (AIMSs) are increasingly being used for anesthesiologic recordkeeping.1 Electronic patient records are considered to be better than handwritten anesthesia records because they require less time and are more complete, accurate, and reliable.2–7 AIMSs provide improved recordkeeping of anesthetic procedures, accurate guiding of patient management, and enhanced patient safety, because the anesthesiologist can focus on intraoperative events instead of manual charting.1,7,8 Additionally, data from AIMSs are a valuable resource for database research9–11 and medicolegal litigations.12–14
Although the quality of data capturing and registering in AIMSs is considered highly accurate, not all stored vital parameter values are based on valid measurements. The course of anesthesia often clarifies the accuracy of a measurement in the clinical situation. However, during data acquisition from the monitoring system, the AIMS cannot verify whether a particular value is a true value or an artifact. As a consequence, retrospectively it is difficult or impossible to differentiate between values representing the actual patient’s vital state and artifacts. Hence, artifacts influence the validity of research based on AIMSs. Furthermore, artifacts may lead to bias, if certain artifacts are associated with specific patient characteristics (e.g., age) or the phase of anesthesia.
Previous research demonstrated a low incidence of artifacts in an AIMS database of an adult population; e.g., 0.3% of all oxygen saturation measured by pulse oximetry (Spo2) values and 24.2% of deviating Spo2 values were caused by artifacts.15 In children, the incidence of artifacts has thus far only been investigated for Spo2 and was higher than in adults; 46% of episodes with an Spo2 of at most 90% occurring in children under anesthesia were artifacts.11
We hypothesized that the difference in physiology and anesthetic technique between children and adults would result in a different incidence of artifacts and deviating values. Therefore, we assessed the validity of AIMS data in a pediatric population undergoing a procedure under general anesthesia and defined factors associated with artifacts.
Materials and Methods
Setting and Study Population
The study was approved by the local Medical Research Ethics Committee of the University Medical Center Utrecht (Utrecht, The Netherlands), which waived the need for informed consent, because subjects were not exposed to a research intervention. According to the requirements of Dutch law, anonymity and confidentiality of routinely collected clinical data were assured.
In this prospective observational cohort study, we included pediatric patients who underwent general anesthesia for noncardiac pediatric surgical or diagnostic procedures in a tertiary pediatric university hospital (Wilhelmina Children’s Hospital, University Medical Center Utrecht, Utrecht, The Netherlands) between May and August 2015. One operating room per day was randomly assigned by drawing paper lots with numbers corresponding with the available operating rooms in sealed envelopes. There were five operating rooms available for randomization. Operating rooms were excluded when there were only cardiothoracic and/or angiographic procedures performed, because of possible preexistent abnormal vital parameter values and/or expected long periods of unreliable vital parameter measurements or when they were located outside the operating complex (e.g., magnetic resonance imaging suite). All anesthetics were observed by the investigator (A.-l.J.H.), a resident anesthesiologist, who was trained for this research by a specialized pediatric anesthesiologist (J.C.d.G.) to identify artifacts. Because a child could be included more than once, the results are reported in relation to the number of anesthetics. An anesthetic was defined as the registration of an anesthetic procedure from induction to the end of anesthesia when the patient left the operating room. All anesthetics were performed by specialized pediatric anesthesiologists.
Patients were monitored during anesthesia with heart rate (HR), Spo2, noninvasive blood pressure, and (not in every patient) invasive blood pressure by an IntelliVue monitoring system (type MP70, X2 multimeasurement module; Philips, Germany) with a built-in filter for artifacts. The HR displayed on the monitor was derived, according to availability, from the electrocardiogram and/or the plethysmograph; when both of these HR values were available, they were displayed next to each other. The HR value was updated after every new measured QRS complex by calculating the mean HR over the last twelve R–R intervals when the HR was at least 50 beats/min or over the last four R–R intervals when the HR was less than 50 beats/min or by averaging the detected arterial pulsations over the last 8 s, respectively. Spo2 was measured by pulse oximetry (Philips FAST) with a taped sensor (OxiMax-P, single patient use adhesive sensor; Covidien-Nellcor, USA).16 Spo2 was displayed on the monitor as the median value over the last 5 s with an update period of 2 s. The mean noninvasive blood pressure was measured by oscillometry and was displayed on the monitor each time it was measured (usually the interval was set at every 5 min).17 The mean invasive blood pressure was derived from the invasive blood pressure curve, which was displayed beat to beat, and was calculated over the last eight beat pressure values.
End-tidal carbon dioxide (ETco2) was measured by a Cicero anesthesia ventilator (Dräger, Germany) by sidestream sampling of carbon dioxide (200 ml/min) in exhaled gas. Carbon dioxide was detected by infrared spectroscopy, and the ETco2 of each breath was displayed on the Dräger ventilator and with a delay of 9 s displayed on the Philips monitor.18
Values from the Philips monitoring system were stored in a locally developed AIMS (AnStat, version 2.0.4, 2015; Carepoint, The Netherlands) that automatically samples data from the monitoring system every 5 s.11,15,19 This AIMS has a low-pass filter that records data in the database every min after filtering has been applied. This filtering implies that the median value per min was calculated and stored for HR, Spo2, and mean invasive blood pressure and that the highest value per min was stored for ETco2. The mean noninvasive blood pressure was recorded every time it was measured without filtering. The HR can be sampled from the electrocardiogram, plethysmograph, and invasive blood pressure in the Philips IntelliVue monitoring system. The HR recorded in the AIMS was primarily derived from the electrocardiogram, but if this value was not available from the monitor, the HR was derived from the plethysmograph. The origin of the HR value was not stored in the AIMS, only the value. Storing the median (HR, Spo2, and mean invasive blood pressure) or highest (ETco2) value/min was considered an effective method to filter for the majority of artifacts, because, compared to the mean value, the median respectively highest value of the 12 values captured per min is not influenced by short-lasting artifacts.11,15
All data for HR, Spo2, ETco2, and mean noninvasive and invasive blood pressure were collected automatically by the AIMS and visualized on a computer screen in the operating room. The data were collected according to standard clinical practice from the time the monitoring system was connected to the patient before induction until the monitor was disconnected when the patient left the operating room. The investigator (A.-l.J.H.) was present in the operating room during all included procedures and compared the monitor with the values that were stored in the AIMS. Each value in the AIMS was inspected and assessed as being an artifact or a valid value. The phase of anesthesia during artifact occurrence and the causes of artifacts were also documented in the operating room. After data collection, the AIMS database was queried to add additional information to the collected data, including the anesthetic technique and whether a value was deviating from a predefined reference range.
The data collection only took into account data that were stored and displayed in the AIMS. Therefore, data filtered out by the Philips monitoring system or by the AIMS were not taken into consideration. An artifact was defined as any value that was judged invalid and/or not reflecting the patient’s current physiologic state, based on the investigator’s (A.-l.J.H.) consultation with the attending anesthesiologist regarding measurements, physiologic state, and observations in the operating room.
In case of discrepancy between the investigator and the attending anesthesiologist, a second investigator (J.C.d.G., pediatric anesthesiologist and primary investigator) made the final decision. The primary rater (A.-l.J.H.) was trained in the specific observation skills by the second investigator (J.C.d.G.) during a period of 2 weeks before the start of the study.
A deviating value was defined as a value outside a predefined reference range. For Spo211,20,21 and ETco2,22–24 fixed reference ranges were used (table 1). ETco2 reference values were calculated from reference values for arterial carbon dioxide, ranging from 35 to 45 mmHg. ETco2 is normally 2 to 5 mmHg lower than arterial carbon dioxide because of mixing of carbon dioxide containing alveolar gas with expired gas empty of carbon dioxide from the anatomical dead space. We calculated ETco2 reference values based on this difference.23,24 For HR, reference values were based on age.25 For mean blood pressure, reference values were based on age and sex.26 We focused on mean noninvasive and invasive blood pressure, because mean blood pressure is considered to be most important under anesthesia in daily care in our hospital.
The primary outcome was the incidence of artifacts for each included vital parameter. Secondary outcomes comprised the incidence of deviating values, and artifacts as a proportion of deviating and nondeviating values. Additionally, we made group comparisons, determining the artifact incidence among the three anesthesia phases and among different age categories. In addition, the causes of artifacts were recorded. The incidences of ETco2 artifacts and deviating values were related to the anesthetic technique in a post hoc analysis.
Sample Size and Statistical Analysis
To perform a sample size calculation, we predefined an artifact incidence of less than 5% to yield valid AIMS data.15,27 We assumed a maximum 4% incidence of artifacts, and we aimed for a 95% CI from 3 to 5% (lower to upper limit), because less than 5% has been considered a valid artifact percentage for AIMS data.15,27 Moreover, for this calculation, we assumed that artifacts within patients were independent. Because mean noninvasive blood pressure is the measurement performed least frequently, the sample size calculation was based on the hypothesis that 4% of the recorded mean noninvasive blood pressure values would be an artifact. The additional assumptions that the mean noninvasive blood pressure would be measured at least every 5 min and a 95% CI would not exceed 5% led to a required minimum of 1,825 mean noninvasive blood pressure measurements or 9,125 min of anesthesia time.
To assess the association between the artifact incidence and the factors studied, a mixed effects model was constructed per determinant (deviating status of measurement, anesthesia phase, age category, or anesthetic technique), including a random intercept per case. This model was compared with an empty model without the determinant, including a random intercept per case, with a likelihood ratio test. Both binomial mixed effects models use a logit link function and were fitted using maximum likelihood estimation. An association between the determinant and the artifact incidence was considered statistically significant if the likelihood ratio test showed a P value of at most 0.05. Statistical analysis was performed with SPSS version 22.0 (IBM, USA) and R version 3.3.2 (R core team 2016).
In this study, we included 136 anesthetics in 132 patients during a cumulative time of anesthesia of 10,236 min (table 2). HR, Spo2, and ETco2 were measured in all anesthetics, whereas mean noninvasive blood pressure was not measured in eight anesthetics (mostly short ear-nose-throat procedures), and mean invasive blood pressure was measured in five anesthetics (table 3). The percentage of artifacts was lowest for HR (0.5%; 95% CI: 0.4 to 0.7%) and Spo2 (1.3%; 1.1 to 1.5%; table 3; figs. 1 and 2). For mean noninvasive blood pressure, the artifact incidence was 5.0% (4.0 to 6.0%). Mean invasive blood pressure (7.3%; 5.9 to 8.8%) and ETco2 (7.5%; 6.9 to 8.0%) showed higher incidences. ETco2 values contained the most deviating values (40.7%; 95% CI: 39.7 to 41.7%), whereas only 3.1% (2.8 to 3.5%) of Spo2 values deviated from the predefined reference range. Deviating values were more often artifacts than values in a normal range (P < 0.001 for every vital parameter), varying between 3.1% (95% CI: 2.1 to 4.4%; HR) and 38.4% (30.3 to 47.1%; mean invasive blood pressure) artifacts among deviating values and between 0.2% (0.1 to 0.3%; HR) and 3.5% (2.7 to 4.5%; mean noninvasive blood pressure) artifacts among nondeviating values (table 3; figs. 1 and 2). Almost all anesthetics showed ETco2 artifacts (94.1%), unlike HR artifacts, which occurred in only 15 (11.0%) of the anesthetics. Only 23.5% of anesthetics demonstrated deviations in Spo2, contrary to more than 50% of anesthetics for all other vital parameters.
For HR, the artifact incidence was higher during the induction phase (1.0%) compared to anesthesia maintenance and emergence (P < 0.001; table 4; fig. 3). For Spo2, ETco2, and mean invasive blood pressure, the artifact incidence was higher during the induction and emergence phases compared to anesthesia maintenance, with the highest incidence for Spo2 (4.6%) and ETco2 (23.6%) during induction (both P < 0.001) and for mean invasive blood pressure during emergence (37.6%; P < 0.001). For mean noninvasive blood pressure, the artifact incidence was not significantly associated with the anesthesia phases.
ETco2 artifacts showed a higher incidence in children up to 4 yr of age (10.3 to 11.3%) compared to older children (4.0 to 5.1%; P = 0.001; table 5; fig. 4). Artifact incidences for the other parameters did not differ significantly between age groups. The artifact incidence was low in all age groups for HR (range: 0.1 to 1.3%; P = 0.170) and Spo2 (1.0 to 3.1%; P = 0.130). The most common causes of artifacts were electrocautery for HR (52.0%), patient movement for Spo2 (40.0%), mask ventilation during induction for ETco2 (40.4%), an oversized pressure cuff for mean noninvasive blood pressure (37.6%), and relocation of the pressure sensor for mean invasive blood pressure (24.2%; table 6).
Because most ETco2 artifacts were related to mask ventilation (table 6), we related anesthetic technique to the occurrence of ETco2 artifacts and deviating values in a post hoc analysis. ETco2 artifacts were more common with an inhalation compared to an intravenous induction (P < 0.001; table 7). Anesthesia maintenance type was associated with ETco2 artifact and deviating value incidences (both P < 0.001). Among anesthesia maintenance types, the ETco2 artifact incidence was highest with mask ventilation (28.1% vs. 2.5 to 8.0% with other maintenance types). Children up to 4 yr of age received fewer intravenous inductions compared to older children (0 to 25.9% vs. 42.3 to 71.4%; P < 0.001).
Each hospital has its unique amount and characteristics of artifacts in its AIMS, which are dependent on the interaction of many factors. The present prospectively studied pediatric population undergoing general anesthesia, with manual artifact data collection live in the operating room, showed that artifacts are present in a substantial number of recordings and that they are associated with deviating status of the measurement, phase of anesthesia, and anesthetic technique.
In addition, a previous study of our group showed that the number and type of artifacts are also dependent on medical specialty and surgical procedure.15 Moreover, there are many other factors that may influence the number and type of artifacts that include clinical practice (e.g., caseload, standard location of the pulse oximeter probe, timing of venous cannula placement, use of an induction room, type of induction and maintenance of anesthesia, delineation of anesthesia phases), workflow (e.g., human errors), specifications, built-in and selected settings and filters of the patient monitor (e.g., the algorithm for calculation of the measurements), filters, sampling rate, and the possibility to edit data in the AIMS, among others.28–30
The measurement algorithms, filters, and system settings of the various monitors and data storage in AIMS have a large influence on the results of the measurements and the incidence of artifacts. The measurement algorithms for saturation and blood pressure measurement and detection of HR vary among available commercial systems and are known to influence the measurement results and the incidence of artifacts.28–30 Publication of these algorithms of all commercial systems is essential for clinical practice and research.
The incidence of artifacts in our setting (varying between 0.5% for HR and 7.5% for ETco2) cannot directly be generalized to other (pediatric) anesthesia settings. However, the concept that electronic data have limited validity is applicable to all systems and hospitals. The limited validity of the data in AIMS should be taken into account not only for research but also when using these data for clinical practice and medicolegal litigations.1,7–14,31
Factors Contributing to Artifacts
Artifacts do not occur at random in AIMS data. Vital parameter values deviating from a predefined reference range were more often artifacts than values in a normal range. In addition, the artifact incidence differed among the three anesthesia phases. In general, the artifact incidence (except for noninvasive blood pressure) was higher during the induction and emergence, compared to the maintenance phase (table 4). During the induction and emergence phases there was more patient movement, contributing to a higher artifact incidence. Most artifacts in ETco2 were related to mask ventilation, which also occurs more often during the induction and emergence phases.
The anesthetic technique influenced the occurrence of ETco2 artifacts (tables 6 and 7). ETco2 artifacts were more common during an inhalation compared to an intravenous induction. Because younger children more frequently received an inhalation induction compared to older children, the ETco2 artifact incidence was also higher in younger age groups. This is also related to the type of airway device. The incidence of ETco2 artifacts during anesthesia maintenance was highest with mask ventilation and in neonates. With mask ventilation, the dead space is larger compared to ventilation through an endotracheal tube or laryngeal mask airway.32 Furthermore, the gradient between ETco2 and arterial carbon dioxide depends on the dead space–to–tidal volume ratio, which is increased at younger age.23,32 Also, because tube size is more difficult to estimate in younger children, transient gas leakage due to inadequate sealing before replacing the tube by a larger one contributed to a higher incidence of ETco2 artifacts. It is known that ETco2 values are more reliable when cuffed instead of uncuffed tubes are used in this young age group.33 Although we routinely use cuffed tubes in younger children (data not presented), ETco2 artifacts due to inadequate sealing occurred in children who were (initially) intubated with an uncuffed tube.
A remarkable number of artifacts appear due to errors on the part of the anesthesiologist, e.g., with the patient not being connected to anesthesia equipment, a wrong-sized blood pressure cuff, or an inappropriately placed invasive blood pressure transducer (table 6). A considerable percentage of these errors are inherent to pediatric anesthesia with a delay in connection to anesthesia equipment in the operating room and difficulties with choosing an appropriate cuff size or are related to the transfer of patients between operating table and bed. These human errors could easily be avoided by the anesthesia staff, by paying attention, and by giving priority to elimination of these errors.
Consequences for Research, Medicolegal Issues, and Clinical Practice
Artifacts represented a substantial proportion of deviating values, which results in erroneous documentation of deviating vital sign values in the AIMS. Artifacts in AIMS data obscure the patient’s true physiologic state and confound the interpretation of abnormal vital parameter values. Knowledge of the incidence and causes of artifacts is important for clinical, research, and medicolegal issues.1,7–14,31 Artifacts can cause an overclassification of deviations when artifacts are within the deviating range, whereas the actual true patient’s vitals are within the reference range, or an underclassification of deviations, when artifacts are within the reference range, but the true values are in the deviating range. Implications of these registered artifact values are expected to be of greater importance for database research and litigation procedures than for clinical practice. In clinical practice, it is possible to recognize and to neglect artifacts in the AIMS in the context of all available information, including the graphical representation of vital parameters on the computer screen and the combination of variables and/or outlying values. On indication, the anesthesia staff can make free text annotations in the AIMS during anesthesia clarifying artifacts, which could be processed to use in database research.
Current artifact filtering methods, which detect outliers or correct for outliers, for example by excluding vital parameter values above and below certain thresholds using an extreme value filter,34 do not sufficiently deal with artifacts. The artifact filtering method in our AIMS, which corrects for outliers by storing the median or highest value/min, is not perfect either. However, by preventing the influence of short-lasting artifacts by using this low-pass filter, we believe that the values in our AIMS are more valid compared to a method of storing a snapshot or mean value.11,15
A possible solution toward a better identification of artifacts in AIMS is an option to allow users to manually (in)validate, remove or change values, or mark values as artifacts.35 Our AIMS does not have this feature, but even with this possibility, artifacts can still be present in the AIMS database. Because manually produced anesthetic records have shown to be less accurate and complete than computer-generated records,2–7 one could argue that manual annotations, validations, and markings of artifacts may also be inaccurate and incomplete, because both are human work.
Another option might be the automatic identification of (potential) artifacts by intelligent filtering during data capturing from the monitoring system. This method can be used to filter out artifacts automatically or for a pop-up that questions whether the value is an artifact, every time a value outside a predefined reference range is identified, which must be answered.31
The ultimate solution would be an AIMS that automatically filters artifacts from true values using an artifact recognition algorithm,36 which might be possible by considering more parameters around the time at which the vital sign is measured. The latter option would be promising, because it could be applied retrospectively as well. Another advantage of an algorithm considering medicolegal issues is that it is more objective compared to manual annotations in the AIMS.
Despite our efforts to classify artifacts objectively, interobserver differences could have influenced our findings, because anesthesiologists could have given different reasons for similar causes of remarkable, outlying, or deviating values. Labeling as artifacts values that are in fact true values, or vice versa, would result in overestimation or underestimation, respectively, of artifact incidences.
In addition, awareness of anesthesiologists of situations that increase the risk of artifacts and deviating values may have been raised by the presence of the investigator in the operating room and by knowledge of the purposes of this study (Hawthorne effect). This could have resulted in behavior avoiding artifacts and deviations and an underestimation of incidences of artifacts and deviations.3,37
Finally, our AIMS documents one single HR value, depending on availability derived from either the electrocardiogram or plethysmograph. It might be possible that the incidence of HR artifacts may be different if each monitoring modality was recorded separately.
The present study showed that vital parameter data in the AIMS are not always valid and hold a considerable number of artifacts. Deviating values are more often artifacts than values in a normal range. Importantly, artifacts do not appear at random in the data, and they are associated with the phase of anesthesia and anesthetic technique. HR and Spo2 have a low incidence of artifacts, whereas mean noninvasive and invasive blood pressure and ETco2 have a higher incidence of artifacts in our pediatric anesthesia practice. Due to differences between hospitals in clinical practice, workflow, monitoring systems, and AIMSs, our findings cannot directly be generalized to other (pediatric) anesthesia settings. However, the concept that electronic data have limited validity is applicable to all systems and hospitals. The present study highlights the awareness for the presence of artifacts in AIMS data of vital parameters and should stimulate researchers in the anesthesia field to take into account artifacts in their research databases. Furthermore, development of (automatic) data validation systems or solutions to deal with artifacts in data is warranted. Independently of the used equipment and procedures, attention for factors associated with artifacts may lead to ways to reduce (the influence of) these factors and eventually a reduction of artifacts in AIMS databases.
Support was provided solely from institutional and/or departmental sources.
The authors declare no competing interests.