Statistical Method to Evaluate Management Strategies to Decrease Variability in Operating Room Utilization : Application of Linear Statistical Modeling and Monte Carlo Simulation to Operating Room Management

To analyze variability in underutilized OR time, we combine (i.e., "bin") data by day. We use the data from each day to develop summary statistics of total times that can be used by OR managers in their decision-making regarding how to decrease day-to-day variability in hours of underutilized time. These daily total times are analyzed by using a system of simultaneous linear statistical equations. The analysis of such a statistical model is also called "structural Equation modeling." The precise statistical model that we develop represents an example that applies principles that are appropriate to the two OR suites that we consider: The University of Iowa's OR and the Ambulatory Surgery Center. Different OR suites with different organizational structures and scheduling practices may require slightly different equations. Appendix 1 includes step-by-step instructions for an OR manager to use our method.

For each surgical case, the data used for the analysis include:

- Date on which the surgical case was started

- Time when the case was started (i.e., the time when the patient entered an OR)

- Time when the case ended (i.e., the time at which the patient left an OR)

- Location (i.e., the OR in which the case was performed)

- Preoperative estimate of case time

- Classification of the case as being elective or add-on

All analyses are performed in units of hours (i.e., an estimated case time of 2 h, 15 min is considered to have an estimated case time of 2.25 h).

The definition of elective or add-on should be entered into the statistical model based on each hospital's work environment. For our two example OR suites, we considered a case to be elective if it was scheduled before noon of the preceding day. This definition was common to the two OR suites that we considered. Add-on cases that could have been safely scheduled on an elective basis, and emergency cases were both lumped into the "add-on" category. OR managers at other suites should use definitions appropriate for their site.

For each day to be analyzed, the data required are:

- Date

- Total number of prime-time hours allocated or assigned for use that day in the OR suite (i.e., in all of the ORs at the location)

- Time at which prime time started that day

- Time at which prime time ended that day

Days analyzed should be limited to those during which elective surgery is performed. For our two examples, we did not consider holidays or weekends.

If a hospital's official definition of prime time differs from the real, operational definition, the latter definition may be most appropriate. For example, if a hospital's policy is that "all elective surgery should be completed by 3:00 P.M." and yet the hospital has a full complement of staff until 6:00 P.M. to do any additional cases that the surgeons may have that afternoon, then the appropriate definition for the time at which prime time ends may be 6:00 P.M.

Using the above 10 data points, we calculated seven summary statistics of daily workload in the OR suite for each of i = 1, 2, …, N days considered in the analysis:

- x^a,i = allocated hours of OR time during prime time for the ith day.

- y^ee,i = estimated hours of elective cases during prime time for the ith day.

- y^e,i = actual hours of elective cases during prime time for the ith day. When calculating this value, cases that started during prime time but finished after the end of prime time contributed only those hours that occurred during prime time.

- y^a,i = actual hours of add-on cases during prime time for the ith day. Cases that started before prime time but finished during prime time contributed to those hours that occurred during prime time.

- y^ae,i = estimated hours of add-on cases during prime time for the ith day.

- y^t,i = hours of turnover time and other delay time during prime time for the ith day. Turnover time is the time from when a patient leaves an OR until another patient enters the same OR, provided the new patient enters during prime time that day. We consider other definitions of turnover time in the Discussion section, "Definition of Turnover Time."

- y^u,i = hours of underutilized time during prime time for the ith day, where y^u,i = x^a,i - y^e,i - y^a,i - y^t,i.

There should be, by definition, no missing values in the analysis (i.e., numbers will be available for all seven variables for each prime-time day).

The number of allocated OR hours (x^a,i) is assumed to be determined completely outside of the model and is therefore said to be "exogenous." x^a,i is equivalent to an independent variable in regression analysis. An exogenous variable functions only as an independent variable and is not influenced by other variables in the model. For both of our examples, The University of Iowa OR suite and the Ambulatory Surgery Center, the number of allocated hours of OR time was established months earlier.

The other six variables are determined jointly and are therefore said to be "endogenous." These dependent variables are represented using a "y." These endogenous variables serve as dependent variables in the model and can also have causal effects on the other endogenous variables. For simplicity, we drop the "i" from the notation whenever it is not relevant to our discussion.

The goal of our analysis is to identify the relative importance of causes for day-to-day variation in hours of underutilized OR time (y^u). To measure day-to-day variation in each of the seven variables, we calculate the variance of each of the variables. For example, the sample variance of Equation 12, where N refers to the number of days considered in the analysis and y^urefers to the sample mean of y^u. To measure the relationships among variables, we calculate their covariances. For example, the sample covariance between actual hours of elective cases (y^e) and hours of underutilized time Equation 13. Among the seven variables, there are seven variances and 21 covariances. Together they form the "sample covariance matrix," examples of which are given in the Results section. As the two equations within this paragraph show, when the variances and covariances are calculated, the mean of each variable is subtracted from the variable. Therefore, the covariance matrix is not affected by the mean values of each of the seven variables over the N days of observation. For all subsequent derivation we can thus consider the mean values to have been subtracted from each of the variables. For example, we write the sample covariance between actual hours of elective cases (y^e) and hours of underutilized time (y^u) as equaling Equation 14, rather than Equation 15. This approach permits us to neglect constant terms from the equations, which makes the derivations briefer. This assumption has no effect on the generality of our results; it just makes the algebra easier. [2]

The structure that we propose for the model is shown graphically in Figure 1. This path diagram represents pictorially the system of simultaneous linear statistical equations. [3] The seven observed variables are represented by the rectangular boxes. Random day-to-day variability in the endogenous (y) variables are introduced into the statistical model by the five random effects [Greek small letter epsilon]. As explained in Appendix 2, the random effects are assumed to follow a normal distribution with a mean of zero. Circles signify these five unobservable random effects (Figure 1).

The exogenous variable x^a(i.e., allocated hours of OR time) has an impact on other variables but is not affected itself by any of the other variables in the statistical model. Therefore, the rectangle surrounding allocated hours does not receive any arrows (Figure 1).

The six endogenous variables (y^ee, y^e, y^t, y^a, y^ae, and y^u) act as dependent variables in at least one dependence relationship. The rectangles that represent these variables all have at least one arrow pointing toward them (Figure 1).

The allocated hours of OR time during prime time for the ith day (x^a,i) can affect, or cause a change in, the number of estimated hours of elective cases during prime time for the ith day (y^ee,i). The word "cause" is a statistical term in this sentence, as the statistical analysis of systems of linear equations or "structural Equation modeling" is also called "causal analysis." The actual hours of elective cases, add-on cases, and turnover time cannot cause a change in the estimated hours of elective cases because the actual hours of cases occur after the estimation of the hours of elective cases. Each 1-h increase in the allocated hours of OR time during prime time for the ith day (x^a,i) can be considered to cause a 1 [middle dot] c^aeeincrease in the number of estimated hours of elective cases during prime time for the ith day (y^ee,i):Equation 1

As described in the section "Generating the Covariance Matrix," the mean value has been subtracted from all of the variables in the linear model (Equation 1). In the sense that c^aeerepresents the slope of the relationship between x^a,i and y^ee,i, if the mean value was not subtracted from the variables in the linear model before being used in Equation 1, then Equation 1would include another coefficient representing the intercept.

The random effect term [Greek small letter epsilon]^eein Equation 1essentially equals 1 [middle dot][Greek small letter epsilon](^ee). The path coefficient of 1 is fixed at this value, which is written next to the corresponding arrows in Figure 2 and Figure 3. The path coefficients specify the relationships shown by the arrows relating the rectangles in Figure 2 and Figure 3.

At hospitals with the same hours of allocated OR time each prime-time day (i.e., with little variability in x^a,i among days), Equation 1could be neglected, and y^ee,i [approximately =][Greek small letter epsilon]^ee,i.

The actual hours of elective cases during prime time for the ith day Equation 2

When the OR is busy, elective cases may start later in the day than planned, be completed in part outside of prime time, and not contribute fully to y^e, the hours of elective cases during prime time. Cases are more likely to be completed entirely during prime time on days when underutilized time is higher than average. We represent this effect in Equation 2using the c^u[arrow right] e y^u,i term.

We could use a three-coefficient model (c^xa[arrow right] e x (^a),i - c^a[arrow right] e y^a,i - c^t[arrow right] e y^t,i) to represent the process of bumping elective cases outside of prime time. The effect of using a three-coefficient model would be to add two coefficients to the statistical model. However, because y^u,i = x^a,i - y^e,i - y^a,i - y^t,i, we are able to use the term c^u[arrow right] e y^u,i to represent the process using one parameter. The one-parameter model is parsimonious. Representing a one-parameter model using c^u[arrow right] e y^u,i is easier to implement in popular statistical packages than setting c^xa[arrow right] e =- c^a[arrow right] e =- c^t[arrow right] e.

To illustrate c^u[arrow right] e mechanistically, we consider a hypothetical OR suite with one OR, which contains on the ith day an add-on case, followed by a turnover, and then followed by an elective case. A 1-h decrease in the allocated hours of OR time (x^a,i), a 1-h increase in the actual hours of the add-on case (y^a,i), or a 1-h increase in the hours of the turnover time (y^t,i) would bump the same number of hours of the elective case outside of prime time.

Adding c^u[arrow right] e y^eto both sides of the top line of Equation 2eliminates the y^eterm from the right-hand side of the equation. The left-hand side of Equation 2then equals (1 + c^u[arrow right] e)y^e. Each 1-h increase in the number of estimated hours of elective cases (y^ee) is considered to cause a mean increase in the actual hours of elective cases equal to 1 [middle dot] c^ee[arrow right] e or a mean increase in the actual hours of elective cases during prime time equal to 1 [middle dot] c^ee[arrow right] e (1 + c^u[arrow right] e)^-1. Likewise, the variance of the differences ([Greek small letter epsilon]^e) between surgeons' estimates and their actual hours of elective cases during prime time equals (1 + c^u[arrow right] e)^-2multiplied by the variance of the differences between surgeons' estimates and their actual hours of elective cases.

The number of actual hours of add-on cases during prime time for the ith day (y^a,i) depends on the number of estimated hours of add-on cases during prime time for the ith day (y^ae,i) as well as the hours of underutilized time during prime time on that day (y^u):Equation 3

The more hours of elective cases and add-on cases on the ith day, the more hours of turnover time (y^t,i) would be expected. Yet, using the same argument made to formulate Equation 2 and Equation 3, the effect of the hours of underutilized time (y^u,i) can also be included:Equation 4

From the top line of Equation 4, the estimated value of c^u[arrow right] t is determined solely by the causal effect of x^aon y^t. For example, each 1-h increase in the number of actual hours of elective cases during prime time (y^e,i) causes (c^e[arrow right] t - c^u[arrow right] t) x (1 + c^u[arrow right] t)^-1increase in the hours of turnover time during prime time.

Emergency cases arrive randomly. Therefore, the estimated hours of add-on cases (y^ae,i) represent, in part, requests made independent of how the OR is functioning that day. Yet, some of the add-on cases are relatively elective. Surgeons may review the day's schedule and submit add-on cases on those days when the estimated hours of elective cases (y^ee,i) seem lower than average. Thus, Equation 5

We also include an Equation thatrestates the definition of hours of underutilized time:Equation 6

The sample covariance matrix is used computationally to find the coefficients in Equation 1, Equation 2, Equation 3, Equation 4, Equation 5, Equation 6and the variances of the five random effect terms (Appendix 3). The statistical algorithm (weighted least squares with the asymptotically distribution-free discrepancy function) varies the values of the coefficients to minimize the discrepancy between the sample covariance matrix and the covariance matrix predicted by the statistical model. [3]

We assess goodness of the fit of the model to the data in several ways (Systat 7.0 for Windows95, SPSS Inc., Chicago, IL). First, we examine the observed (sample) covariance matrix and the residual matrix (i.e., the actual covariance matrix minus the predicted matrix). Second, we consider whether the magnitude and sign of each estimated coefficient is reasonable mechanistically. [2] Third, we test how well the statistical model with the estimated coefficients would fit the population or "true" (underlying) covariance matrix, if it were available, using the root mean square error of approximation. [4] The statistic reflects differences between elements of the population covariance matrix and corresponding elements of the best fit of the model to the population covariance matrix. We also calculate, as is recommended, [4] the 90% confidence interval for the statistic.

Prime-time days with values that are distant from those of other days are referred to as "outliers." An outlier day that leads to substantial changes in the sample covariance matrix and subsequently in estimated coefficients of the statistical model would be considered to be "influential." To screen for influential observations, we substitute the estimated coefficients and each day's exogenous variables into Equation 7, Equation 8, Equation 9, Equation 10, Equation 11given in Appendix 3 to calculate each day's random effect terms. We apply Hadi's multivariate outlier detection algorithm [5] to these random effect terms (Systat 7.0 for Windows95, SPSS Inc.). This algorithm looks at all of the random effect terms simultaneously to Figure outwhich days are unusual. The outlier days are deleted, and a new set of coefficients are estimated for the statistical model. [3] After confirming that outlier days are not influential, the original sets of coefficients using all data are used in the following Monte Carlo simulations.

We next use the estimates of the coefficients in Equation 1, Equation 2, Equation 3, Equation 4, Equation 5, Equation 6and variances of the five random effect terms to evaluate the effect that management interventions could have on day-to-day variability in underutilized OR time. In particular, we use Monte Carlo simulation to predict the effect that eliminating each random effect term, combination of terms, or day-to-day variability in allocated hours (the exogenous variable) would have on day-to-day variation in underutilized time. Eliminating each term (i.e., setting it equal to 0) represents mathematically the impact of a perfectly effective management intervention. We express each result as the percentage decrease in the standard deviation of underutilized time that would be achieved by the intervention. The methodology for the Monte Carlo simulations is described in Appendix 4.

Estimates of the coefficients and variances of the linear statistical model will not perfectly match the true (underlying) values of the coefficients and variances in the model. Variability in the estimates of the coefficients and variances may affect the results of the Monte Carlo simulations. To assess this possibility, we performed the following statistical analysis for our two example OR suites:

1. We created a new hypothetical data set of N days by randomly sampling with replacement from the N days of data included in the original analysis. Each random selection of one of the i = 1, …, N days obtained that day's measurements of the seven variables (x^a,i, y^ee,i, y^e,i, y^t,i, y^a,i, y^ae,i, and y^u,i).

2. We estimated the coefficients in Equation 1, Equation 2, Equation 3, Equation 4, Equation 5, Equation 6and the variances of the five random effect terms using the new hypothetical data from step 1.

3. We performed the Monte Carlo simulations using the estimated coefficients and variances from step 2 and calculated the percentage decreases in the standard deviation of underutilized time that would be achieved by each management intervention.

4. We repeated steps 1–3 nine times.

5. From step 4 we had 10 estimates of the percentage decrease in the standard deviation of underutilized time that would be achieved by each management intervention. We calculated for each intervention the standard deviation of the 10 estimates.

The standard deviation of the estimates provides insight into the precision of the estimate of the percentage decrease that would be achieved if the management intervention was performed.

We obtained, from the patient tracking databases, six data points for each surgical case and four data values for each day of elective surgery in 1996, as described in the Methods section "Obtaining Data from an Operating Room Information System." For The University of Iowa OR, these data included 11,579 cases each with six data values and 244 days each with four data values. There were 11,579 x 6 + 244 x 4 = 70,450 data values. From these values and using the definitions given in the Methods section, we calculated the seven variables of interest, giving 244 x 7 = 1,708 numbers. The seven variables are summaries of daily workload in the OR suite. The covariance matrix of the seven variables contains 28 numbers (Table 1). For the Ambulatory Surgery Center, the data included 4,842 cases and 244 days, giving 4,842 x 6 + 244 x 4 = 30,028 data. Using the definitions given in the Methods section, "Obtaining Data from an Operating Room Information System," we calculated the seven variables. This step gave 244 x 7 = 1,708 numbers, which we used to calculate the covariance matrix (Table 1).

The covariance matrices provide some insight into what aspects of OR management are likely to affect day-to-day variation in underutilized time. The upper and lower panels of Table 1(y^u[h of underutilized time]) show the covariances between hours of underutilized time and each of the other six summaries of daily OR workload. The positive covariance between hours of underutilized time and allocated hours of OR time implies that each increase in allocated hours results in an increase in hours of underutilized time. The negative covariances between hours of underutilized time and estimated hours of elective cases, hours of elective cases, hours of turnover time, hours of add-on cases, and estimated hours of add-on cases imply that each increase in these hours results in a decrease in hours of underutilized time. The magnitudes of the covariances between hours of underutilized time and hours of elective cases (175 h²and 26.6 h²for The University of Iowa OR suite and the Ambulatory Surgery Center, respectively) and the magnitudes of the covariances between underutilized time and estimated hours of elective cases (146 h²and 19.4 h², respectively) exceed the magnitudes of the covariances between underutilized time and the other four variables. The magnitudes of the covariances therefore suggest that management interventions to decrease day-to-day variation in underutilized time would have the greatest effect if they were to focus on improving methods to (1) predict how long elective cases will actually last and (2) select on which day to do each elective case so as to best fill the allocated hours. However, the magnitudes of the covariances do not provide quantitative estimates for the decreases in the standard deviation of underutilized hours that each management intervention can achieve. Quantitative estimates can help OR managers balance the predicted cost of implementing a management intervention versus the predicted financial gain to the OR suite.

For each example, from the (1) definitions given in the Methods section, (2) 28 numbers in the sample covariance matrix (Table 1) calculated from the definitions, and (3) system of simultaneous linear statistical equations or "structural Equation model" (Figure 1), we calculated 14 terms (Figure 2 and Figure 3). The coefficients in Equation 1, Equation 2, Equation 3, Equation 4, Equation 5, Equation 6are written next to their corresponding arrows. The variances of the random effect terms are given within the circles.

We evaluated how well the statistical model fit the data. First, the maximum absolute difference between the actual covariances (Table 1) and the predicted covariances equaled 6.9 h²and 1.4 h²for the University of Iowa OR suite and the Ambulatory Surgery Center, respectively. Second, the signs of the estimated coefficients (Figure 2 and Figure 3) were as they should be based on Equation 1, Equation 2, Equation 3, Equation 4, Equation 5, Equation 6. Third, the magnitudes of the estimated coefficients were reasonable. For example, surgeons at The University of Iowa OR suite had a tendency to underestimate their case times. Referring to Equation 2and Figure 2, an increase of 1 h in estimated hours of elective cases caused a 1 [middle dot] c^ee[arrow right] e = 1.4-h increase in the actual hours of elective cases and a 1 [middle dot] c (^ee)[arrow right] e (1 + c^u[arrow right] e)^-1= 1.4 (1 + 0.4)(^-1)= 1.0-h increase in the actual hours of elective cases during prime time. Fourth, a root mean square error of approximation of 0.05 or less indicates a close fit between the statistical model's predicted covariance matrix and the population (i.e., true or underlying) covariance matrix. [4] A value of 0.08 or less indicates a reasonable fit. [4] We obtained 0.03 for The University of Iowa OR (90% CI, 0.0–0.05) and 0.03 for the Ambulatory Surgery Center (90% CI, 0.0–0.07).

We evaluated, for The University of Iowa OR, the effects of four possible modifications of the statistical model (Equation 1, Equation 2, Equation 3, Equation 4, Equation 5, Equation 6) on the residual covariance matrices and the room mean square error of approximation. First, we considered neglecting paths producing reciprocal relationships in the system of linear statistical equations. Paths shown in Figure 1 and Figure 2from underutilized hours to elective hours, add-on hours, and turnover hours were deleted. The root mean square error of approximation equaled 0.19 versus 0.03 noted previously. Values exceeding 0.1 are considered to imply that the statistical model is unacceptable. [4] The maximum absolute difference between the actual covariances (Table 1) and the predicted covariances equaled 39 h²versus 6.9 h²noted previously. Second, paths shown in Figure 1 and Figure 2from underutilized hours to elective hours, add-on hours, and turnover hours were deleted. A path from elective hours to add-on hours was added. A path from add-on hours to elective hours was added. The root mean square error of approximation equaled 0.16. The maximum absolute difference between the actual and predicted covariance 37 h (²). Third, paths shown in Figure 1 and Figure 2from underutilized hours to elective hours, add-on hours, and turnover hours were deleted. Three paths were added, each from allocated hours to elective hours, add-on hours, and turnover hours. The root mean square error of approximation equaled 0.16. The maximum absolute difference between the actual and predicted covariances equaled 21 h². Fourth, we added four coefficients to the statistical model (Equation 2, Equation 3), as described in the Methods section immediately after Equation 2. The root mean square error of approximation equaled 0.0 versus 0.03 in the preceding paragraph. Model fit to the data was not significantly improved (chi-square difference test, P = 0.13). The maximum absolute difference between the actual and predicted covariances equaled 4.4 h²versus 6.9 h²noted previously.

Prime-time days with values that are distant from those of other days are called "outliers." There were 1 and 3 outliers for The University of Iowa OR and the Ambulatory Surgery Center, respectively, out of the 244 days of elective surgery in 1996. For example, on the outlier day from The University of Iowa OR, there were an unusual number of add-on cases during prime time, including an organ harvest, two liver transplantations, and an exploratory laparotomy. We repeated the statistical analysis without these three outlier days of data. Estimated coefficients and variances (Figure 2 and Figure 3) differed by less than 0.02 and 0.3 h², respectively. These three outlier days were included in all subsequent analyses.

Operating room managers can measure variability in underutilized time for an OR suite, but what is essential is to know how to intervene to decrease the variability in underutilized time. We used the values (Figure 2 and Figure 3) from the statistical modeling to evaluate the effects that eliminating day-to-day variation in the allocated hours or random effects could have on day-to-day variation in underutilized time (Table 2). Management implications were the same at The University of Iowa OR and the Ambulatory Surgery Center. Results may be different at other OR suites.

Interventions to eliminate day-to-day variation in estimated hours of elective cases, excluding that accounted for by variation in allocated hours, are predicted to be able to cause the largest absolute and percentage decrease in the standard deviation of underutilized hours. Thus, the decisions that have the greatest impact on underutilized time are the decisions as to the days on which to do each of the elective cases. Such decisions are made days to months before the days at which they affect the hours of underutilized time. Inaccuracy in the surgeons' estimates for how long their cases take is not the major problem. Management interventions that would be performed on the day of surgery are predicted to have little effect on variation in underutilized OR hours. For example, the successful elimination of all variability in turnover time and unexpected delays between cases is predicted to decrease the standard deviation of underutilized time by less than 2%. The successful elimination of all variability in estimated hours of add-on cases each day could be done most easily at the Ambulatory Surgery Center, where add-on cases tend to be elective cases added the afternoon before the day of surgery. However, achieving this goal would decrease the standard deviation of underutilized time by less than 5%. The combination of interventions that is predicted to be able to cause the greatest decrease in the variation in underutilized hours is to eliminate variation in (1) estimated hours of elective cases and (2) actual hours of elective cases.

If an OR suite routinely has some days for which the elective cases are scheduled to be completed by 11 A.M. and other days for which cases are to be completed at 4 P.M., then it is unlikely that any intervention on the day of surgery will consistently increase OR utilization (decrease underutilized time). Causes of underutilized OR time (i.e., poor OR labor productivity) that can be adjusted on the day of surgery would all be less important than the method of selecting the day on which to do each elective cases so as to best fill the allocated hours.

We show that linear statistical modeling and Monte Carlo simulation can be used to evaluate management strategies to decrease day-to-day variability in hours of underutilized OR time. Our strategy for data reduction is valuable if OR patient tracking systems are to be used in management decision-making. Our analysis reduced 70,450 and 30,028 data values in patient tracking databases to 14 numbers (Figure 2 and Figure 3). Each of these 14 numbers gives information to the administrator or physician managing an OR suite. However, to develop figures such as Figure 2 and Figure 3, which the OR manager can use on a quarterly basis, we found it necessary to make compromises. For example, we excluded consideration of the discrete nature of surgical cases. As a result, reasons for day-to-day variation in underutilized time cannot be traced back to individual surgical cases or ORs.

An advantage of the method that we use to define turnover time is in its implications for calculating the cost of underutilized time. We define turnover time to be the time from when a patient leaves an OR until another patient enters the same OR, provided the new patient enters during prime time that day. As a result, underutilized time in each OR always occurs at the end of a series of cases. The elimination of day-to-day variation in underutilized OR time would result in underutilized time being the same every day. Staff could then be rescheduled, resulting in no underutilized time each day. Another way of defining turnover time is to have a maximum turnover time between two consecutive cases. For example, some OR suites set the turnover time to equal 1 h whenever the actual time between two consecutive cases is greater than or equal to 1 h. This definition of turnover time may provide for a more realistic estimate of underutilized OR time. Nevertheless, subject to this definition of turnover time, eliminating day-to-day variation in underutilized time may not be sufficient to permit staff to be rescheduled. As a result, the logic in analyzing causes of variability in underutilized time so as to increase labor productivity is lost.

The disadvantage in how we define turnover time is that we cannot distinguish in our analysis between cleaning and set-up time versus avoidable or unavoidable delays. Interpreting the time interval from when one patient leaves an OR to the time when the next patient enters that room as turnover exaggerates the time actually used to perform the needed cleanup and preparation performed between the two cases. However, making a distinction between times for cleaning and set up versus delays will not affect managerial decision-making for the two examples that we present. At both of the example OR suites, eliminating all variability in hours of turnover time (resulting from cleaning, set up, or delays) is predicted to decrease variability in underutilized time by less than 2%(i.e., will not significantly decrease labor costs). At the two OR suites considered as examples in this article, management interventions focused on decreasing variability in the time required to complete nursing and anesthesia activities on the day of surgery will not decrease costs. However, at an OR suite with substantial unproductive time in the mornings on some days, as might occur if a surgeon arrives on some but not all afternoons to begin operating at a "second site," how we defined turnover time could contribute to inappropriate conclusions.

A concern in using structural Equation modelingis how to determine a reasonable sample size. When characterizing the relationships among variables, sample sizes fewer than 100 days are small. [3] There are large improvements in statistical power when the number of days is increased from 100 to 200. Beyond 200, each incremental addition tends to have less of a benefit. Sample sizes exceeding 400 or so are considered to be large. [3] For the two OR suites that we considered, elective surgery is scheduled on 244 days each year, and so 1 yr of data is a reasonable choice.

The statistical analysis of systems of linear equations or "structural Equation modeling" can be used to analyze data from OR patient tracking databases to identify the cause(s) of variations in daily OR workload. Monte Carlo simulation can then be used to predict the effects that management interventions could have on variation in daily OR workload. Together, these analyses can provide to the OR manager increased understanding as to what should be priorities for intervention to decrease underutilized OR time, increase the productivity of the OR suite, and substantially decrease perioperative costs. In our application of the analyses, we found that at both example OR suites eliminating all (1) errors in predicting how long elective or add-on cases would last, (2) variability in turnover or delays between cases, or (3) day-to-day variation in hours of add-on cases would not decrease variability in underutilized OR time by more than 10%. Our analysis found, for these two OR suites, that these commonly cited causes of poor OR productivity are all less important than the method of selecting the day on which to do each elective cases so as to best fill the allocated hours.

1. As described in the Mathematics section "Obtaining Data from an Operating Room Information System," choose appropriate definitions of cases as "elective" versus "add-on" based on the OR suite's policies. Choose appropriate definitions of the scheduled hours of operation for elective cases. Have a programmer extract the six data points for each surgical case from the OR information system. Have a programmer extract the four data values for each day from the OR information system. The time required for step 1 will depend on the sophistication of the OR suite's information system. At The University of Iowa the data were stored in disparate databases on an IBM mainframe computer, and so downloading the data to an Access (Microsoft, Redmond, WA) database took approximately 1 month.

2. From the six data points for each surgical case and four data values for each day, calculate the seven variables for each day. Choose an appropriate definition of "turnover time," as considered in the Discussion section "Definition of Turnover Time." Queries need to be written by a programmer in the language of the database used to store the data. Our experience has been that this step takes a programmer experienced with surgical services data approximately 2 weeks.

3. Import the data into a statistics package that can calculate multivariate outliers, such as SYSTAT or SAS. Have the statistics package generate the covariance matrix, and use whatever statistical method comes with the package to identify multivariate outliers. This step takes approximately 1 h. Based on the results and the rationale described in the Results section "Obtaining Data from an Operating Room Information System," the OR manager needs to decide whether to proceed with the full analysis. If so, and if the OR manager does not have previous experience with structural Equation modelingand Monte Carlo simulation, we recommend that the OR manager consider using a consultant statistician. The time required for the remaining steps will depend on the complexity of the linear statistical model. Our experience has been that to complete the project takes approximately 1 month. Calculations take a few minutes on a Pentium computer.

4. Formulate a linear statistical model that is appropriate for the OR suite and with sufficiently few coefficients that they can all be estimated. Equation 1, Equation 2, Equation 3, Equation 4, Equation 5, Equation 6provide a reasonable starting point. This step requires close interaction between the OR manager and the statistician.

5. Add assumptions about the random effect terms that are appropriate for the OR suite. We discuss some relevant issues in Appendix 2.

6. Use a statistics program to estimate coefficients of the linear statistical model. We consider several appropriate programs in Appendix 3.

7. Use the output provided by the statistics program to assess goodness of the fit of the model to the data, as described in the Methods section "Estimating Coefficients of the Statistical Model" and Results section "Linear Statistical Modeling of Day-to-day Variability in Hours of Underutilized Time." If necessary, return to step 4 and modify the model.

8. Check for days that are outliers, and repeat estimation of the coefficients if needed.

9. Use the coefficients in the Monte Carlo simulations. We consider statistical programs to perform this step in Appendix 4.

The five random effect terms are analogous to the residual terms in ordinary linear regression. We assume that [Greek small letter epsilon](^ee),i, [Greek small letter epsilon]^e,i, [Greek small letter epsilon](^t),i, [Greek small letter epsilon]^a,i, and [Greek small letter epsilon]^ae,i (1) follow a normal distribution with a mean of zero, (2) are not correlated with the exogenous variable in the model (allocated hours [x^a]), (3) are not correlated with one another, and (4) are not correlated with random effects on other days. These assumptions are some of the assumptions of routine, ordinary linear regression. These assumptions are not typically made as part of structural Equation modeling, but are included in our model because they are reasonable for the OR suites and simplify the Monte Carlo simulations that will be described. With respect to the third assumption, let us suppose that on days when an OR suite is busy that the method of collecting time data could become inaccurate. For example, this may happen if nurses record time data for each surgical case on paper, and then the numbers are later typed into a computer by a secretary. Then, if one random effect term were different from the mean because the number was recorded inaccurately, the other numbers would be more likely to differ from their mean values because they too are likely to have been recorded inaccurately. The random effect terms would be expected to be correlated with one another. Therefore, OR managers should be particularly attentive to checking this assumption if they expect there to be large measurement error in the variables. [3] The fourth assumption implies that what happens during prime time on one day does not affect what happens on the next day. For example, having one day with many estimated hours of elective cases should not increase the probability that the next prime-time day has many estimated hours of elective cases (i.e., the covariance of [Greek small letter epsilon]^ee,i and [Greek small letter epsilon]^ee,i+1 should equal zero). If these assumptions are unreasonable, routine methods exist for OR managers to make suitable modifications of the linear statistical model (e.g., as described in the book by Long [2]).

The sample covariance matrix is used computationally to find the coefficients in Equation 1, Equation 2, Equation 3, Equation 4, Equation 5, Equation 6and the variances of the five random effect terms. We use the method of weighted least squares with the asymptotically distribution-free discrepancy function (Systat 7.0 for Windows95, SPSS Inc., Chicago, IL). [3,6] This method can permit combinations of coefficients to be constrained to nonzero constant values (as in Equation 6) and also does not assume that the data follow a multivariate normal distribution. [3,6] We perform the analysis in a sequential fashion. First, we assume that C^u[arrow right] e = C^u[arrow right] t = C^u[arrow right] a = 0 to simplify Equation 2, Equation 3, Equation 4in a manner to permit ordinary linear regression to estimate the coefficients of the statistical model. Second, we use the resulting coefficient estimates as initial conditions in the final analysis including all coefficients in Equation 1, Equation 2, Equation 3, Equation 4, Equation 5, Equation 6.

We have repeated the analysis for The University of Iowa OR suite using some other structural Equation modelingprograms to check our results. To estimate coefficients using the analysis of moment structures (AMOS 3.61b, SmallWaters Corporation, Chicago, IL), hours of underutilized time should be represented as an unobserved variable with a zero error variance. AMOS produces an adequate fit ([chi squared][6, n = 244]= 8.55; P = 0.20; RMSEA = 0.04). The parameter values are similar to those shown in Figure 2. A LISREL VI emulation of this model produces an identical result. The value of Mardia's measure of multivariate kurtosis is small (-0.58) and not significantly different from zero, suggesting the distributional properties of the six observed variables are excellent. Readers interested in the details of performing these estimations in SYSTAT, AMOS, or SAS may contact the first author.

For some OR suites it may be possible to use ordinary linear regression to estimate coefficients in the statistical model. At The University of Iowa OR, add-on cases accounted for a mean of 16% of the hours of cases. Some of these add-on cases were emergencies that postponed elective surgery until later in the day. On days when there were few hours of underutilized time, hours of elective cases were thus decreased (Figure 1 and Figure 2). At the Ambulatory Surgery Center, add-on cases accounted for 6% of the hours of cases and tended to be "elective" cases added the afternoon before the day of surgery. In contrast to the results from The University of Iowa OR, at the Ambulatory Surgery Center underutilized time had no significant effect on hours of elective cases (Figure 3). As a result, for the Ambulatory Surgery Center, paths shown in Figure 1 and Figure 3from underutilized hours to elective hours, add-on hours, and turnover hours could have be deleted. Ordinary linear regression would then be adequate to estimate coefficients and calculate variances of the statistical model (Figure 1). This would permit all of the analyses presented in this paper to be done with Excel and an add-in statistical package such as @Risk (Palisade Corporation, Newfield, NY). The analyses would be much simpler to implement.

We rewrite Equation 1, Equation 2, Equation 3, Equation 4, Equation 5, Equation 6for use in the Monte Carlo simulations. Substituting Equation 6into Equation 1, Equation 2, Equation 3, Equation 4, Equation 5and rearranging terms, Equation 7, Equation 8, Equation 9, Equation 10, Equation 11

For each simulated day, the software running the Monte Carlo simulation (@Risk, Palisade Corporation) generates six random numbers: one for each of the five random effect terms and the exogenous variable (allocated time). The random effect terms are normally distributed random numbers with means equal to zero and variances as represented by the linear statistical model. The exogenous variable (allocated OR hours) is represented using its empirical distribution function [7](ExpertFit version 1.21, Averill M. Law & Associates, Tucson, AZ). [parallel] The random numbers are substituted into Equation 7, Equation 8, Equation 9, Equation 10, Equation 11, and the system of simultaneous linear equations is solved by matrix inversion and multiplication (Microsoft Excel 97). Underutilized time for the day is calculated using Equation 6. The Monte Carlo simulation software then proceeds by generating a new set of six random numbers for the next simulated day. Underutilized time for that day is calculated. The process is continued until each additional 100 simulated days changes the standard deviation of underutilized time by less than 0.5%. [7] We have also performed this analysis using SAS to check our results. Readers interested in SAS computer code may contact the first author.

[parallel] For example, at The University of Iowa OR suite, the prime-time day ran on Mondays and Tuesdays from 8:00 A.M. to 3:30 P.M. and on Wednesdays, Thursdays, and Fridays from 7:00 A.M. to 3:30 P.M. As a result, the empirical distribution function essentially equaled one value on 2 of 5 or 40% of days and a different one on the other 3 of 5 or 60% of days.