Anesthesia information management systems allow automatic recording of physiologic and anesthetic data. The authors investigated the prevalence of such data modification in an academic medical center.
The authors queried their anesthesia information management system database of anesthetics performed in 2006 and tabulated the counts of data points for automatically recorded physiologic and anesthetic parameters as well as the subset of those data that were manually invalidated by clinicians (both with and without alternate values manually appended). Patient, practitioner, data source, and timing characteristics of recorded values were also extracted to determine their associations with editing of various parameters in the anesthesia information management system record.
A total of 29,491 cases were analyzed, 19% of which had one or more data points manually invalidated. Among 58 attending anesthesiologists, each invalidated data in a median of 7% of their cases when working as a sole practitioner. A minority of invalidated values were manually appended with alternate values. Pulse rate, blood pressure, and pulse oximetry were the most commonly invalidated parameters. Data invalidation usually resulted in a decrease in parameter variance. Factors independently associated with invalidation included extreme physiologic values, American Society of Anesthesiologists physical status classification, emergency status, timing (phase of the procedure/anesthetic), presence of an intraarterial catheter, resident or certified registered nurse anesthetist involvement, and procedure duration.
Editing of physiologic data automatically recorded in an anesthesia information management system is a common practice and results in decreased variability of intraoperative data. Further investigation may clarify the reasons for and consequences of this behavior.
ANESTHESIA information management systems (AIMS) allow automated recording of physiologic and anesthesia care data. AIMS create a more complete and accurate record of such data compared with a traditional handwritten record, especially at the beginning and end of cases, when more attention to patient care (rather than documentation) is required.1,2Despite this and other benefits, only a small number of centers use an AIMS. One purported impediment to AIMS acceptance is a fear of increased liability due to automated inclusion of transient or artifactual data.3In a survey of AIMS administrators, the majority of respondents indicated that users at their centers are allowed to edit data that have been automatically recorded from patient monitors.4There is no information on the incidence of the editing of electronically acquired data during AIMS use. Therefore, we investigated the extent to which such data modification occurred in one academic medical center where an AIMS has been in use for many years.
Materials and Methods
Our academic medical center began using an AIMS (CompuRecord; Philips Medical Systems; Andover, MA) for intraoperative record keeping in 1991 and has expanded the system over time to include workstations in approximately 50 anesthetizing locations. Physiologic and anesthetic parameters (e.g. , blood pressure, fraction of inspired oxygen) are automatically acquired by the AIMS workstations from clinical monitors. Values are recorded every 15 s for continuously measured parameters (e.g. , oxygen saturation measured by pulse oximetry [Spo2]) and less frequently for intermittently measured parameters (e.g. , noninvasive blood pressure). Parameters available from multiple/redundant sources simultaneously (e.g. , pulse rate [PR] from pulse plethysmography, peripheral arterial catheter, pulmonary artery catheter) are all recorded and annotated with the identity of the source device.
Acquired data may be displayed graphically or numerically on the AIMS video display during the case and on the printed record after the case ends. The choices of which parameters to display/print and the time scale to use are set by departmental defaults but may occasionally be changed by individual users. The system displays/prints only the median value of each chosen parameter for each epoch of raw data (usually 2 min). Parameters not chosen for the primary display or printed record are still permanently stored in their raw form in the AIMS database vital sign log and can be reviewed during a case or anytime after a case ends.
Invalidation of any automatically recorded value (data point) can result from several processes:
Preset limit filters: The AIMS contains configurable preset ranges of clinically appropriate numerical limits for each parameter (e.g. , systolic blood pressure [SBP] between 25 and 250 mmHg). Values outside of these ranges are presumed to be artifactual. Values received by the AIMS that are outside of the preset ranges are automatically flagged as invalid and are suppressed from display/printing. The user may revalidate data points that have been automatically set to invalid by this filtering mechanism. Individual users with advanced knowledge of the AIMS configuration options may alter these preset limits at any time, but our experience is that no one does so to any significant extent.
Data stream toggling: The data stream for each automatically recorded parameter can be toggled on or off by clicking a virtual button on the display screen. This causes the values in that stream to be flagged as invalid immediately upon acquisition. This may be used preemptively in situations when artifactual values are anticipated. Although this is frequently done during cardiopulmonary bypass, our experience suggests that this toggling feature is rarely used in other settings.
Tabular data review: Any recorded data point (or group of data points) in a case can be manually invalidated by opening the case’s tabular vital sign log, highlighting the target values, and flagging the values as invalid.
Any user (attending, resident, or certified registered nurse anesthetist [CRNA]) at an AIMS workstation with a case in progress can invalidate (or revalidate) any recorded value. Our AIMS does not record the identity of the user who invalidated any data point, nor does it differentiate among invalidation methods (data stream toggling or tabular data review), though it does identify values that are filtered out by preset limits. Our department does not have a policy regarding the invalidating/editing of case data, though records become locked 30 min after the case is ended, and no further changes can be made after that time. Data that remain invalidated will be suppressed from the AIMS video display and printed record and may appear as a gap in monitoring. Regardless of which invalidation process occurs, however, the originally captured (raw data) value remains permanently stored in the electronic case record and can be retrieved if needed. An additional feature of our AIMS is that any invalidated data point can optionally be appended by manual entry of an alternate value.
After obtaining institutional review board (Program for the Protection of Human Subjects, Mount Sinai School of Medicine, New York, New York) approval for a retrospective investigation and waiver of informed consent, we queried our AIMS database of all cases performed in 2006. Cases involving cardiopulmonary bypass were excluded because of the long periods of time when data streams are toggled off (as described in the list above). Records of anesthesia for labor and delivery (but not cesarean deliveries alone) were also excluded because physiologic data during such cases are not recorded automatically in our AIMS. Data points automatically invalidated by preset limit filters (as described in the list above) were excluded from analysis because they were not deliberately invalidated by users. Data points actively/deliberately invalidated by users (via data stream toggling or tabular data review) were included.
For all cases analyzed, the total count of raw, nonnull values recorded for each parameter was tabulated, as was the state of each value (i.e. , valid or invalid). For each data point, the phase of anesthetic care (i.e. , before, during, or after the surgical/diagnostic procedure) was determined, as was the presence or absence of an intraarterial monitoring catheter. For all invalidated data points, the presence or absence of a manually entered alternate value was noted, and the difference between the two was calculated. Also calculated was the count of and duration of time between clusters (contiguous data points) of invalidated values. Additional practitioner and patient characteristics of interest were also extracted for each case and were set as attributes of every data point within that case.
Descriptive statistics were calculated for the data set. For each case with any invalidated data, the variance of each physiologic parameter for the entire case was calculated both with and without the invalidated data included. For several select parameters, the probability of data invalidation was modeled by the method of generalized estimating equations for Poisson regression using SAS version 9.1 (SAS Institute, Inc., Cary, NC). This method provides relative risks of a data point being invalidated in the presence of various patient/practitioner/procedure-related factors relative to the absence of that same factor, and independent of (i.e. , controlling for) the state of all other included factors considered simultaneously. Factors included in the model were as follows: extreme values (the ranges of which were defined specifically for each parameter), participation of a resident or CRNA, American Society of Anesthesiologists (ASA) physical status classification, case emergency status, and procedure duration. This method also controlled for correlated data for individual attending physicians and provides robust standard errors for hypothesis testing. The type 3 tests based on the Wald statistics were used to test the overall significance of a variable. P < 0.05 was considered statistically significant.
A total of 29,491 cases were included. Of these cases, 5,593 (19%) had one or more data points invalidated. There were 294,362,643 automatically recorded values, 238,731 (0.08%) of which were invalidated. Table 1shows the total count of values for each parameter, the proportion of those values that were invalidated, and the distribution of all invalidated values across the 24 automatically recorded parameters. For cases with any invalidated data, there was a median [25th–75th % interquartile range (IQR)] of 3 [1–3] parameters invalidated per case.
A group of 58 attending anesthesiologists (from a total of 72), each of whom had worked without a resident or CRNA in at least 10 cases and performed a median of 52 cases (range, 10–234), was identified. For this group, data were invalidated in a median [IQR] of 7% [2–15%] of each attending’s cases, with 52 (93%) of the attending physicians invalidating data in at least one case.
Additional analyses were performed for three physiologic parameters that comprised a high proportion of all invalidated values: SBP, PR, and Spo2. SBP was derived from noninvasive oscillometric blood pressure cuff or intraarterial catheter sources, and PR was determined primarily from plethysmography or intraarterial catheter sources (rather than from electrocardiography that is recorded separately as “heart rate”). Figures 1A–Cshow histograms of invalidated values for these three parameters.
Systolic Blood Pressure
At least one invalidated SBP value was found in 10% of cases, with a median [IQR] of 2 [1–6] invalidated values per case. These invalidations resulted in a decrease in total SBP variance in 92% of cases. Manually entered values were provided to replace 7% of all invalidated SBP values. The median [IQR] difference between the invalidated values and the manually entered replacement values was 4 mmHg [−8 to 24 mmHg].
To characterize the clustering of invalidated data, SBP measured by intraarterial catheter was chosen as a parameter with predictable and frequent (every 15 s) acquisition. For 1,069 cases with SBP measured by intraarterial catheter and with more than one invalidated value, the invalidated data were found in clusters of an average of 8 values (2 min) each. For 74% of the cases with more than one cluster of invalidated values, there was a median [IQR] of 2 [1–4] clusters per case and 3 [1–13] min between clusters.
The relative risks (95% confidence intervals) of data invalidation for SBP less than 80 mmHg or SBP of 160 mmHg or greater (vs. 80–160 mmHg) were 25.8 (24.1–27.6) and 7.1 (6.6–7.7), respectively. There was a 2.2 (2.1–2.4) and 2.2 (1.8–2.6) relative risk of SBP data invalidation for preprocedure values versus both intraoperative or postprocedure values, respectively. SBP values obtained from an arterial catheter carried a 0.21 (0.19–0.23) relative risk of invalidation (vs. noninvasive blood pressure). Greater ASA physical status classification, emergency status, and longer procedure duration were all independently associated with increased SBP invalidation (data not shown).
At least one invalidated PR value was found in 9% of cases, with a median [IQR] of 6 [2–18] invalidated values per case. These invalidations resulted in a decrease in total PR variance in 97% of cases. Manually entered values were provided to replace 13% of invalidated PR values. The median [IQR] difference between the invalidated values and the manually entered replacement values was −1 beats/min [−58 to 2 beats/min].
The relative risks of data invalidation for PR less than 60 beats/min or PR of 120 beats/min or greater (vs. 60–119 beats/min) were 2.3 (2.2–2.4) and 14.8 (13.6–16.0), respectively. There was a 6.2 (5.8–6.8) and 9.4 (7.2–12.3) relative risk of PR data invalidation for preprocedure values versus intraoperative or postprocedure, respectively. PR values obtained from an arterial catheter carried a 1.5 (1.4–1.6) relative risk of invalidation. Resident and CRNA presence conferred a 1.4 (1.2–1.7) and 1.7 (1.4–2.1) relative risk, respectively, of PR data invalidation (vs. no resident or CRNA involvement). Greater ASA physical status classification, emergency status, and longer procedure duration were all independently associated with increased risk of PR invalidation.
At least one invalidated Spo2value was found in 4% of cases, with a median [IQR] of 6 [3–12] invalidated values per case. These invalidations resulted in a decrease in total Spo2variance in 95% of cases. Manually entered values were provided to replace 35% of invalidated Spo2values. The median [IQR] difference between the invalidated values and the manually entered replacement values was 6% [3–10%].
The relative risk of data invalidation for Spo2less than 90% (vs. 90% or greater) was 115 (105–126). There was a 2.4 (2.1–2.7) and 2.6 (2.2–3.1) relative risk of Spo2data invalidation for preprocedure and postprocedure values, respectively, versus intraprocedure values. Both greater ASA physical status classification and emergency status were independently associated with increased risk of Spo2invalidation.
Our results suggest that manual invalidation of automatically recorded data in an AIMS is a common practice at the study institution, having occurred in 19% of all anesthetics during the period analyzed. When working unassisted, 93% of attending anesthesiologists edited data in at least one of their cases. The combined predominance (59%) of PR, SBP, and Spo2among invalidated data is not surprising, because these parameters are prominently displayed graphically on our AIMS workstations, so deviations in these parameters are more likely to be noticed than other parameters that are less visible. In nearly all cases, these invalidations resulted in “smoothing” of the anesthetic record (seen as a decrease in variance).
It is not surprising that only a minority of invalidated values were appended with manually entered replacement values. In cases where valid adjacent values were present in the vital signs log, there would be no need for manual replacement, because a median value would still be displayed for the recording epoch for that variable. When no valid contemporaneous values were available, there would be no source from which to obtain the replacement value that would avoid a gap in the anesthesia record vital signs. In the absence of alternative data sources, it is likely that manually entered values were estimated based on clinical examination or interpolated from the adjacent values.
We found that data invalidation was independently associated with several factors, though our retrospective methods cannot prove causality, nor can we elucidate why editing occurred because the reasons for doing so are rarely documented or are explained in text narratives that are problematic to analyze. We are not surprised, however, that extreme values were more likely to be invalidated than are values within normal ranges, because such values may be perceived by practitioners as creating medicolegal risk and/or truly artifactual values may be more likely to be outside of normal limits. The increased invalidation during the preprocedure period compared with later phases may relate to an increased focus of attention on charting early in the case at a time when other administrative data are typically entered in our AIMS. Alternatively, practitioners may believe that physiologic disturbances in the preprocedure phase are more reflective of anesthesia care, compared with later events that may be related to the surgery/procedure. The higher likelihood of invalidation during longer procedures could be due to the increased amount of time during which practitioners have the opportunity to review and edit previously acquired data. The higher likelihood of data invalidation in higher ASA classification patients could be due to more concern regarding the consequences of brief disturbances in high-risk patients. The presence of a resident or CRNA may increase invalidation simply because more than one anesthesia care team member is attending to the electronic record. The finding that intraarterial catheters were associated with more frequent PR invalidation but fewer SBP invalidation events are likely related to the different patterns of artifact that occur in comparison with noninvasive blood pressure measurement using current hemodynamic monitoring systems.
Filtering of data is routine during the creation of handwritten anesthesia records and results in the familiar “railroad track” appearance of many such records.1This may result from clinicians omitting values they interpret as artifactual, omitting values thought to be too transient to be clinically significant, not reporting values that occur between fixed recording intervals (e.g. , every 5 min in a handwritten record), and noncontemporaneous recording of values based on recall (with potential bias). There may also be omission or modification of values that are perceived to reflect questionable anesthesia care quality or that may be perceived to increase medicolegal exposure.3Although clinicians may smooth the record in an AIMS to portray a more stable anesthetic, evidence from our previous work suggests that this effect is less pronounced compared with handwritten records.2In addition, we found that many of the invalidated values lie within “normal” physiologic ranges. This may be due to efforts to smooth data even within normal ranges, deleting blocks of data that included normal values along with extreme values, or preemptively toggling data streams off while acquired values still remained accurate. Previous investigations have demonstrated that extreme values may be omitted from handwritten records.1,2We are unaware of published reports regarding frequency and predictors of manual data editing in an AIMS with which to compare our findings.
The problem of artifacts from clinical monitors is well described, and technological solutions have been proposed.5,6Until such filtering mechanisms are improved significantly and integrated into practice, however, clinicians will likely continue to modify AIMS data. Although several hospitals do not permit editing of automatically recorded data, most facilities with AIMS allow at least some editing.4In the presence of an electronic audit trail, such editing is consistent with the standard practice for paper medical records, whereby erroneous entries are crossed out and initialed.
Standards for both paper and electronic medical records require audit trails such that changes are documented. We interpret this principle to indicate that AIMS should retain the original values permanently in the underlying databases. Most clinician users are probably unaware of the existence of audit trails. Given the legal discoverability of an audit trail, however, it may be prudent for clinicians to document the reasons for the alterations—a function that most AIMS do not currently facilitate. From a medicolegal standpoint, we know of no evidence that routine editing of data is superior to defending a claim that artifacts and transient readings are frequent occurrences that usually lack clinical significance.
We conclude that editing of data automatically recorded in an AIMS is common practice at a university medical center. In the majority of circumstances, these edits reduced the variability of the intraoperative data. Extreme values, phase of the perioperative period, care team mix, ASA physical status, emergency procedures, presence of an intraarterial catheter, and procedure duration all influenced the likelihood of data editing in an AIMS. Further investigation is warranted to determine whether there is medicolegal significance to these editing practices.