Operating Room Utilization Alone Is Not an Accurate Metric for the Allocation of Operating Room Block Time to Individual Surgeons with Low Caseloads

As we have done in other studies, 2,5,6we designed a mathematical computer model to “act like” a surgical suite with respect to surgical case scheduling. Simulated cases were scheduled into OR block time. After simulating the creation and scheduling of thousands of hypothetical cases, OR utilization was calculated. Mathematical details of the computer simulations are given in the  Appendix.

From previous work, we knew many of the factors that tend to increase the number of weeks of data required to estimate OR utilization accurately. 2We deliberately designed our computer simulations to model hypothetical conditions under which the average OR utilization can be estimated more precisely than it can be under “real world” conditions. By doing so, we assured that our study underestimated the actual widths of confidence intervals (CI) for the average OR utilization.

Specifically, our simulated surgeons never took vacations, got sick, expanded their practices, lost patients, or changed the days of the week when they operate. Thus, the average OR caseloads of the surgeons in the simulation model remained the same for decades. That had the effect of artificially reducing the widths of CIs for the average OR utilization.

To further make sure that our study underestimated the actual widths of CIs for the average OR utilization, we programmed every patient and surgeon to arrive for surgery as scheduled. The intensive care unit, wards, and postanesthesia care unit were never full. Necessary equipment was available for every case. 7,8No case was ever canceled.

The baseline computer simulations of case scheduling had the following assumptions. We comment on them in the Limitations section of the Discussion.

The number of workdays between requests of successive patients to be scheduled for surgery was exponentially distributed. 9The exponential distribution is characterized by the mean number of patients each week who request surgical care. The use of the exponential distribution for patient arrivals implies that the timing of a patient request for surgery is not affected by how many days the patient expects to wait to have surgery. It also implies that the decision of one patient to be scheduled for surgery does not affect the decision of another patient to be scheduled for surgery.

Each case was assigned a time duration generated randomly from a log-normal distribution with mean ± SD = 3.79 ± 2.36 h. 2,3,5This log-normal distribution describes the durations of cases performed at the main surgical suite of the University of Iowa. 5These cases are a mixture of routine and specialized procedures, performed on an inpatient, same-day admit, or outpatient basis. The lower and upper quartiles of case duration were 2.2 h and 4.7 h. By using historical case durations to schedule cases, we assured that the cumulative hours of underutilized OR time (i.e. , the time that a case is completed early) was approximately equivalent to the cumulative hours of overutilized OR time (i.e. , time that a case is completed late). 10,11 

Scheduled turnover times (“patient out” to “patient in”) were 30 min.

All surgeons were allocated one 8-h block of OR time each week.

When a patient requests to be scheduled for surgery, the case is scheduled to be performed on the earliest possible date with sufficient open block time for the case.

Patients balk (i.e. , leave the queue for surgery) if they have to wait more than 4 weeks for surgery. This means that they either receive care at a different surgical suite, with a different surgeon, or do not have surgery.

Each computer simulation used a different mean number of patients requesting each week that they be scheduled for surgery (assumption No. 1). Each computer simulation produced 10,000 3-month or 1-yr periods.

We computed the OR utilization for each of these 10,000 periods. The average of the 10,000 measurements of OR utilization was taken as the average OR utilization (i.e. , the expected value of OR utilization). This average OR utilization was plotted on the vertical axes of the figures. The 2.5th, 5th, 95th, and 97.5th percentiles of the 10,000 measurements were plotted along the horizontal axes of the figures. The 2.5th and 97.5th percentiles provided 95% CIs for the average OR utilization. The 5th and 95th percentiles provided 90% CIs for the average OR utilization.

Auto-correlation is the measure of the degree of correlation between successive measurements. We measured the correlation between the OR utilizations of successive blocks for 50,000 consecutive blocks. Note that, based on assumption No. 4, consecutive blocks are equivalent to consecutive weeks.

We repeated the simulations using the mean numbers of patients requesting to be scheduled for surgery that achieved correlation coefficients of 0.3 and 0.1, respectively.

Dependence of autocorrelation on average utilization can be accounted for by the two sources of random variation in the simulations: random variation in the numbers of patients each week requesting to be scheduled for surgery and random variation in case durations. To evaluate which source of random variation was more important, we assessed the sensitivity of our results to the mean and the SD of case duration. We repeated the analyses while assigning every case a fixed duration of 3.75 h. Since the simulated surgeon had one 8-h block each week (assumption No.4 above), the surgeon could perform 0, 1, or 2 cases weekly. Utilizations of each block could be 0% ([0]÷ 8 h), 47% ([3.75]÷ 8 h), or 100% ([3.75 + 0.5 + 3.75]÷ 8 h). Finally, we assigned every case a fixed duration of 25 min and used a turnover time of 5 min.

The absolute width of the CI for the average OR utilization is important because it affects the utility of the OR utilization metric for purposes of allocating OR block time. We found that each reduction in the measured OR utilization caused an increase in the width of the 95% CI.

With 3 months of data, the differences between the upper and lower limits of the 95% CIs exceeded 10% for average adjusted utilizations of 90% or less (fig. 1).

With 1 yr of data, the differences exceeded 10% for average adjusted utilizations of 87% or less (fig. 2).

The results were similar for raw utilization. With 3 months of data, the differences exceeded 10% for average raw utilizations of 83% or less (fig. 3).

Suppose that, during a 3-month period, two surgeons have adjusted utilizations measured at 65 and 80%, respectively. An OR manager or surgical committee needs to decide whether to reduce the block time of the surgeon with the 65% adjusted utilization to give the time to the surgeon with the adjusted utilization of 80%. Analysis of the computer simulations used to create figure 1revealed that there is at least a 16% chance that the surgeon with the lower measured utilization does not actually have a lower average utilization (i.e. , that the difference was due just to random error).

Correlation coefficients between utilizations of successive blocks varied depending on the average utilization. The autocorrelation was low (e.g. , less than 0.1) when the average utilizations were relatively high (e.g. , adjusted or raw utilizations exceeding 87 and 81%, respectively) (fig. 4).

In figure 5, all simulated cases had the same case durations, but the CIs were wider than in figure 1. Consequently, the predominant cause of autocorrelation and wide CIs was random variation in the numbers of patients each week requesting to be scheduled for surgery. At the higher OR utilizations, random variation in case duration affected whether a case could be scheduled into any one block. This reduced the autocorrelation (fig. 4) and thus reduced the widths of the CIs (figs. 1 and 5).

Simulations with cases of 25-min durations (fig. 5) show that our results are sensitive to the number of patients who can have surgery in OR block time each day. Average OR utilization can be estimated more precisely when more patients have surgery in OR block time each week.

Confidence intervals for the average OR utilizations are wider than OR managers may predict intuitively. This is partly because successive measurements of OR utilization are correlated with one another. Although increasing the amount of historical data from 3 months to 1 yr decreases the statistical uncertainty about a surgeon's average utilization, CIs remain wide. Most importantly, each reduction in the measured utilization causes an increase in the width of the CIs.

Thus, OR utilization is the least accurate for precisely those surgeons for whom it is likely to matter. For example, suppose that, during one quarter, the measured adjusted utilization is 65% for a surgeon. Then, the average adjusted utilization could be as low as 38% or as high as 83% for that surgeon. With 1 yr of data, the average adjusted utilization for that surgeon ranges between 50 and 78%.

Our results show that 3 months or 1 yr of OR caseload data alone are insufficient to estimate raw and adjusted utilizations precisely enough for use in making statistically sound data-driven decisions on OR block time allocation for individual surgeons. The problem is not in trying to estimate average OR utilization precisely per se , but in trying to do so for individual surgeons rather than groups of surgeons.

Our results are important because the allocation of OR time to individual surgeons using only historical OR utilization, as the data-driven component of that decision, is a widespread practice. At many facilities, the OR manager may claim that the block time is being allocated to surgical services/groups, not to individual surgeons. However, if the services then simply allot their OR time by surgeon, then the OR time has in effect been allocated to the surgeons.

Most surgical suites in the United States perform all cases scheduled by its surgeons, provided a case can be done safely. The cases may need to be scheduled as “add-on” or called “urgent,” but the cases are done. At such a surgical suite, OR time is allocated not because the surgical suite has only a fixed number of hours of OR time available, but as a way to care efficiently for all of the patients undergoing surgery. 12 

If too much OR time is allocated, then utilization is low, which reduces OR efficiency (i.e. , the weighted sum of underutilized and overutilized OR time). 12,13If too little OR time is allocated, then the staff work late, which also reduces OR efficiency. 12,13The allocation of OR time providing the optimal balance maximizes OR efficiency. Methods are available to accurately calculate the optimal OR allocation, whether the surgeon and patient choose the day of surgery 12–17or whether all of the surgeons’ patients receive care within a predetermined reasonable number of days. 2,6,14,18,19The latter methods 2,6,14,18,19reduce uncertainty in OR utilization, caused by the autocorrelation described in the current paper, while still including some OR time allocated to individual surgeons. These methods to determine optimal OR allocations to maximize OR efficiency are statistically reliable, practical, and not related to measuring OR utilization. 2,12–18By using these methods, the problem described in this article is avoided entirely. We recommend allocating OR time based on OR efficiency.

Some administrators are reluctant to use these methods 2,6,12–19for outpatient and same-day-admit surgery patients at facilities with a fixed, externally determined, annual budget (e.g. , in Canada or a county hospital in the United States). Some of these facilities really may have fixed hours of OR time, because otherwise they would run a deficit. For facilities truly having fixed hours of OR time, methods have been developed to allocate OR block time and schedule cases while considering restrictions on the facility budget, 7,20,21even with limits on hospital bed availability. 7,21–24Factors that can be included in the analysis, in addition to OR utilization, are strategic objectives of the facility, future externally determined annual budgets, and productivity. For example, block time can be allocated not by determining the time needed for all of the cases for one surgeon, 2but by planning enough block time to perform most of the cases within the regular block time for that one surgeon. 2As many as possible of the remaining cases are performed in other OR time scheduled on a first-come–first-serve basis, either allocated to all surgeons or to services. 6,19That way, the surgical suite gets a high OR utilization and surgeons get some individually allocated OR time. These methods 2,6,7,20–24differ from the current study in that the OR management goals are not to allocate the right amount of OR time to each surgeon. Instead, the goals (for good or for bad depending on the perspective) are to maximize surgical suite objectives at the expense of some surgeons who may lose OR time based on random error.

We recommend that OR utilization alone not be used to allocate OR block time to individual surgeons because that application of it is unreliable statistically (figs. 1–3). There are four other important reasons why we recommend that OR utilization not be used for this purpose.

First, allocating OR block time based on utilization does not consider the resulting effect on hospital ward and intensive care unit bed availability. 7,21–24For example, at hospitals with 100% midnight occupancy of ward and intensive care unit beds, it make sense to preferentially increase OR block time allocations to surgeons who perform mostly outpatient surgery.

Second, the hospital contribution margin (revenue minus variable costs) achieved from allocated OR time can vary several hundred percent among surgeons with the same OR utilization. 7,20The hospital may benefit from allocating more OR block time to surgeons who achieve a high contribution margin per hour of OR time.

Third, hospital variable costs resulting from each hour of OR time varies several hundred percent among surgeons. 20,21Suppose that more OR time is allocated to a surgeon with a relatively high OR utilization but also high variable costs per allocated hour of OR time. Then, allocating more OR time to that surgeon may result in higher hospital variable costs. At hospitals with a fixed annual budget, this may result in deficit spending.

Fourth, efforts to increase OR utilization can reduce the operating margin of a hospital. 5For example, a hospital may sign more reduced fee-for-service contracts to increase the number of patients that can receive care at the hospital. But, each increase in the percentage OR utilization can be at the expense of a larger increase in the average patient waiting time, prompting patients to seek care elsewhere. The net effect of signing more discounted contracts to increase patient volume can be a net reduction in the average revenue per case and overall hospital perioperative margin. 5 

Figures 4 and 5provide an understanding of the results shown in figures 1 to 3. Understanding the reason for our results is important, because this is how it is known that the results can be generalized to many surgical suites.

Figures 4 and 5show that the case duration, the variation of duration in each case from its scheduled duration, the turnover time, the duration of OR block time, etc. , are unlikely to have important effects on our conclusions other than to the extent that they influence the average number of cases done per week.

As explained in the Results, figures 4 and 5show that our results are sensitive to the pattern and frequency of arrival of requests from patients to be scheduled for surgery. In fact, it is the combination of the surgeon and patient together that matters, in that for convenience, we modeled the surgeon as never taking vacation or getting sick. Nonetheless, we assumed that there is never a change in an average OR caseload for a surgeon or in the days of the week when the surgeon operates at the surgical suite. We also assumed that every patient arrived as scheduled, the intensive care unit was never full requiring case postponement, and necessary equipment was available for every case. As explained in the Methods, to the extent that, by design, none of these assumptions reflect the extent of the variance in the real world, the results of figure 4 and 5suggest that our CIs in figures 1–3are likely narrower than in the real world.

Average OR utilization is least likely to be estimated precisely for a surgeon who does one or two long cases each week, but often changes his or her day of surgery because of other commitments, skips one week but operates twice the next week, and so forth. This pattern likely arises most often at academic medical centers. Therefore, our results may be most important for academic practices.

Average OR utilization is most likely to be estimated precisely for a surgeon with many very short cases and a high OR utilization. This would be analogous to a surgical clinic or office-based practice. Utilization may be a good basis for planning clinic-based resources. The experience of surgeons and hospital administrators in clinic management is not analogous to OR management, because many surgical suites average only two cases per OR per day. 25 

The sensitivity of our results to the pattern and frequency of arrival of patient requests to be scheduled for surgery represented a challenge to us in designing a final set of simulations for presentation. The strategy we chose (see Methods) was to systematically underestimate the widths of CIs for the average OR utilization. This has a clear drawback. Although we increased the likelihood that our conclusions are sound for surgeons with low caseloads, we do not know the magnitude of our safety factor. We do not know how to determine how many cases per OR per day that a surgeon needs to do in order for utilization to be measured sufficiently precisely for reasonable management decision-making. We know that it is more than a couple of cases a week. However, whether a reasonable number is four, eight, etc. , is not evident from our study.

We evaluated whether average OR utilization can be estimated precisely for a surgeon who performs a few cases a week in his or her 8 h of allocated OR time. We showed that for such a surgeon, neither average adjusted nor raw OR utilization can be estimated precisely using routine statistical methods with either 3 months or 1 yr of OR block time utilization data. We recommend that, at surgical suites trying to allocate OR time to a surgeon with a low volume of surgical cases, OR management decision-making not be based only on the simple average of historical utilizations. We recommend using the statistically sound alternative of allocating OR time based on OR efficiency. 12–17 

We wrote the computer code in Excel Visual Basic 6.0 (Microsoft, Redmond, WA) to perform each simulation in the following sequence.

The computer used a random number generator 3to generate the length of time (e.g. , 0.5 days) until the next patient requests to be scheduled for surgery, as specified by assumption No. 1 (see Methods).

The computer used a random number generator to generate the duration of the new case from step 1, using assumption 2. Scheduled case durations were right-truncated at 8 h (assumption 4), so that the cases could be scheduled at the simulated surgical suite.

Using the time remaining in each future block, the duration of the case from step 2, and the turnover time from assumption 3, the computer determined the available surgical date for the new case (assumption 5). If there was insufficient time for the case within 4 weeks, then the patient was considered to balk (assumption 6).

After one quarter or 1 yr of simulated time has passed, the average utilization during the preceding quarter or year was calculated.

If fewer than 10,010 quarters or years were simulated, the simulation returned to step 1.

The last 10,000 of the 10,010 quarters or years were analyzed as described in the Methods section “Analyzing simulation output…” The first 10 quarters or years were ignored for analysis, to eliminate the effect of the start-up conditions on simulation results.