The medical, social, and economic effects of the teaching mission on delivery of care at an academic medical center (AMC) are not fully understood. When a free-standing private practice ambulatory clinic with no teaching mission was merged into an AMC, a natural experiment was created. The authors compared process measures across the two settings to observe the differences in system performance introduced by the added steps and resources of the AMC's teaching mission.
After creating process maps based on activity times realized in both settings, the authors developed discrete-event simulations of the two environments. The two settings were comparable in the levels of key resources, but the AMC process flow included three residents/fellows. Simulation enabled the authors to consider an identical schedule across the two settings.
Under identical schedules, the average accumulated processing time per patient was higher in the AMC. However, the use of residents allowed simultaneous processing of multiple patients. Consequently, the AMC had higher throughput (3.5 vs. 2.7 patients per hour), higher room utilization (82.2% vs. 75.5%), reduced utilization of the attending physician (79.0% vs. 93.4%), and a shorter average waiting time (30.0 vs. 83.9 min). In addition, the average completion time for the final patient scheduled was 97.9 min less, and the average number of patients treated before incurring overtime was 37.9% greater.
Although the teaching mission of the AMC adds processing steps and costs, the use of trainees within the process serves to increase throughput while decreasing waiting times and the use of overtime.
What We Already Know about This Topic
Academic clinical care is thought to be more costly than clinical care in private practice
The impact of the academic mission on performance in a private practice pain clinic is not known
What This Article Tells Us That Is New
When simulating the two practices, the academic pain clinic model improved performance metrics but required more processing
RECENT and ongoing efforts to reform health care in the United States are likely to increase pressure to improve the efficiency of delivery of both care and education in academic medical centers (AMCs). For example, enactment of P.L. 111–148, the Patient Protection and Affordable Care Act, is expected to lead to increased patient loads as a result of expanded health insurance coverage. Legislation such as P.L. 111–152, the Healthcare and Education Affordability Reconciliation Act, will motivate AMCs to increase the size of residency programs to handle the increasing demand for healthcare providers. The simultaneous quest for increased efficiency in the delivery of care and increased efficiency in the training of new physicians will make decision-making in the AMC even more complex. One fundamental question on the minds of managers, administrators, public policy experts, and academicians is, what impact does the academic mission of the AMC have on the performance of the systems that deliver patient care? The current work is one effort to address this question.
One common approach to the study of this topic has been to compare the fees charged in AMCs to those charged in private practices (PPs).1,–,6Such studies routinely conclude that patient care in AMCs is more expensive. If the AMC is delivering educational services in addition to patient services, the finding of additional costs is not surprising. Intuition suggests that measurements such as patient flow times, waiting times, and the use of overtime should also be adversely affected by the AMC's educational mission.
The ideal approach to this question would be to conduct a series of experiments to systematically isolate the impact of adding specific educational activities to an existing patient-care delivery system. Unfortunately, as a general rule this is not possible. An alternate approach is to conduct such experiments in a virtual environment. This method introduces many difficulties regarding design of a simulation to include all relevant resources, policies, and such. Our work uses a hybrid approach. We take advantage of a natural experiment, the merger of a PP ambulatory pain clinic with no teaching mission into an AMC with a substantial teaching mission. The two settings were comparable because both clinics operated in the same metropolitan area and had the same clinical director and clinical structure (four examination rooms, one attending physician scheduled, and one physician's assistant). We then created discrete-event simulations of both settings using actual activity times observed in each clinic. This process allowed us to consider the two settings as though they had identical patient loads and schedules over an extended period of time.
Materials and Methods
Goals of This Investigation
The primary goal of this investigation was to evaluate the impact of a teaching mission on the performance metrics of a clinical practice. To do so we considered a pair of outpatient clinics that focus on pain management. A PP with no teaching mission treated new and returning patients who were experiencing acute or chronic pain. This practice subsequently was closed and merged with an ongoing pain-treatment clinic within an AMC. This merger created a natural experiment to compare performance metrics across the two settings. A second goal of this research was to demonstrate how discrete-event simulation of process flows can be used to gain such insights. Although this tool has a long history of use in modeling outpatient clinics,7,–,11to our knowledge it has not been applied explicitly for comparing process performance between AMCs and nonacademic PPs.
Our study centered on the comparison of performance metrics for two settings. The metrics considered included average flow time measured in minutes and average throughput measured in patients per hour. We also tracked utilization levels of examination rooms, residents, the attending physician, and the physician's assistant. Patient-related metrics included average patient waiting time; average number of patients completed by the end of the morning session at 1200 h; likelihood of completing all patients scheduled by 1200, 1230, and 1300 h; the average completion time of the last patient seen; and the average waiting time of the last patient seen.
The performance metrics considered in this work are naturally sensitive to resource levels as well as to patient arrivals. Although an AMC contains a plethora of resources absent from a small PP, we restricted the analysis to the consideration of patients arriving according to a predetermined schedule for consultation with the attending physician or physician's assistant. We did not address the role of the clinic's resources in dealing with inpatient consultations or scheduled procedures. For the processes considered here, the levels of all key, identifiable resources were the same across the two clinics, other than those involved in the teaching mission. In addition, we made the predetermined schedule identical for both clinics in the simulation.
The measured performance differences were driven by two main differences in the processes involved. First, differences emerged because the steps involved in the two processes differ. The process in the AMC includes steps related specifically to the educational mission that are absent in the PP. Typically, the resident reviews each case, visits with the patient, and consults with the attending physician before the attending physician actually sees the patient. Second, even when activities are common between the clinics, some activity times are not identically distributed across the two settings. Most significantly, the attending physician spends more time in direct contact with the patient in the PP. In the AMC, the resident gathers basic information and in many cases develops some insights about the case before the attending physician becomes involved. This process often changes the nature of the interaction between the attending physician and the patient, especially for relatively simple cases.
A discrete-event simulation generates a set of time epochs at which an entity, such as a patient, resident, physician, or examination room, changes states or moves from one step in the process to another.12By tracking each change, the simulation represents the activities of the clinic. Prior models of this type have assumed that processing times are either exponentially or normally distributed to be consistent with basic queuing models and/or to be easily understood by practitioners. Unfortunately, both assumptions are fundamentally flawed. Actual processing times are never distributed exponentially because infinitely long processing times are physically impossible. When processing times in a clinic become exceedingly long, patients can be rescheduled, assistance can be sought from another physician, or patients can be admitted to the hospital for longer-term care. In addition, using an exponential distribution as an approximation of the processing times observed in this system would overstate the likelihood of activity times very close to 0 and yield misleading results. In addition, processing or activity times cannot be normally distributed because the normal distribution allows negative as well as infinitely long processing times.
In addition, the fact that the schedule is created in 4-h blocks guarantees that the system never reaches steady state. Consequently, previous results obtained by applying queuing theory13,14directly to evaluate system performance are not applicable to the settings in question. We developed process maps for each patient-related activity that takes place between a patient's arrival and his/her departure. We then converted the two process maps into computer simulations using ExtendSim, version 7.0 (Imagine That Inc., San Jose, CA).15
Type 1: patients who come to the clinic for an initial visit.
Type 2: patients who come for a follow-up visit and need to be seen by the attending physician.
Type 3: patients who come for simple services such as prescription refills and typically are handled by a physician's assistant.
In simulating these systems, particular attention must be paid to the processing of type 3 patients. In both clinics, these patients are directed to a specific examination room managed by the physician's assistant. This distinction is important because the flow of patients is split after step 2, with type 1 and type 2 patients routed to one of the three remaining examination rooms. The clinic operates from 0800 to 1200 h and from 1300 to 1600 h on weekdays.
Process Flow in the AMC
The process flow of the AMC is depicted in figure 1. The first patient is scheduled for 0800. Each patient registers upon entering the clinic (step 1). A clinical assistant retrieves the patient after an examination room becomes available. Once the patient is in the examination room, the clinical assistant records the patient's vital signs (step 2). For patient types 1 and 2, the resident is then notified of the patient's presence. The resident reviews the patient's file before entering the room (step 3). The resident's time alone with the patient is step 4. After step 4, the resident consults with the attending physician. This is step 5. The attending physician then enters the examination room and interacts with the patient (step 6). After completing the visit with the attending physician, the patient proceeds to checkout (step 7). Finally, the patient exits the system. Key resources used in these processes include the clinical assistant (in step 2), the resident/fellow (in steps 3, 4, 5, and 6), the attending physician (in steps 5 and 6), the examination rooms (in steps 2, 3, 4, 5, and 6), and clerical staff (in steps 1 and 7).
We note that although type 1 and 2 patients have essentially the same sequence of activities, the distribution of the activity times differs. Because type 1 patients are initial visits, the average processing times and variances for steps 3, 4, 5, and 6 are greater than for type 2 patients. We also note that a delay often occurs before the start of step 2 because the clinical assistant may not immediately escort the patient to an available room. To reflect this potential delay, a variable labeled “retrieve patient” was added to the model. Because we did not formally measure this delay, the clinic's director provided estimates of distribution parameters for this time.
For type 3 patients, steps 1 and 2 are identical to those of the other patients, but the third step is initially handled by the physician's assistant. However, cases exist in which the physician's assistant needs to consult with the attending physician before releasing the patient to checkout. In these instances (estimated to be 50% of all cases), this step is added to the process flow. These interactions tend to be quite brief, and parameters of the distribution of this activity time were based on estimates provided by the clinic's director.
PP Pain Clinic
The process flow of the PP clinic (fig. 2) before the merger generally paralleled that of the AMC pain clinic but did not include a teaching component. Consequently, steps 3, 4, and 5 of the process flow shown in figure 1are absent. Physician time with the patient had a mean duration of 28.8 min, compared with 10.7 min in the AMC. This differential is attributable to the resident in the AMC assuming some of the face-to-face patient interaction of a private clinic physician.
We made estimates of the parameters of the activity time distributions based on two data sources. The staff and attending physician at the PP clinic routinely recorded times of each step. Data gathered between March and May 2009 were used to describe activities in this setting. Data regarding processing times at the AMC for all patients seen during February and March of 2010 were gathered by paid observers.
All activity times were treated as samples from a four-parameter β distribution. Each β distribution is defined by the sample minimum, maximum, mean, and variance. The β distribution is continuous between the minimum and maximum activity times. Consequently, generating values for the simulations required the calculation of two shape parameters, α and β. We chose to use the method of moments method to define these parameters as described in Gupta and Nadarajah.16This method guarantees that the minimums, maximums, means, and variances of the actual activity times recorded are all replicated by the variables used in the simulations. Tables 1and 2show parameters of the activity time data for the AMC and PP, respectively. For each variable measured, the tables show the minimum, maximum, mean, and variance of the sample data. The tables also show the α and β values calculated for the variables used in the simulation. Figure 3shows a histogram of actual times for step 6 in the AMC for type 1 patients. The probability density function of the approximating β distribution derived from analyzing the same data is shown in figure 4.
Because type 1 patients require more time for several steps when compared with type 2 or 3 patients, the AMC clinic adopted a scheduling rubric (fig. 5) intended to stagger the cases in such a way as to make the workload of the attending physician more evenly distributed throughout the morning session. To facilitate a comparison between the two environments, we simulated an application of this schedule for both clinic sites. This scenario was repeated to simulate 1000 4-h blocks of service to create measurable results with narrow confidence intervals.
The results of the simulations are reported in tables 3through 6. Table 3reports several performance metrics for the AMC. The row labeled “Base case” refers to simulation results that most closely match the current clinical setting (three residents and four examination rooms). These results are consistent with observations of actual clinic operations shown in the second row, labeled “Actual.” For example, the average flow time for the simulated AMC practice was 76.2 min, whereas the average observed flow time was 79.2 min. We note that the data gathered at both clinics included both appointment times and patient arrival times. The arrival times were used to calculate actual waiting and flow times.
Table 3displays results when the number of residents is cut from three to two in the row labeled “2 Residents.” In this case, average flow time is increased by 11.8 min, whereas the average throughput drops from 3.5 patients per hour to 3.1. We also see that the average waiting time increases from 30.0 to 41.7 min. The next row of the table shows the results of increasing the number of examination rooms from four to five. Although the room has the highest utilization among the resources tracked in the base case, adding an additional room has little impact on system performance. This finding is reasonable given that the one physician's assistant and three residents can simultaneously occupy no more than four rooms. Consequently, the current process cannot use a fifth room effectively.
Table 3also shows results when the teaching time labeled step 5 is changed to 80, 90, 110, and 120% of its original value. The average teaching time within this model is 6.6 min, which is consistent with published studies of anesthesiology residents training in the operating room of a large AMC.17
Table 4presents additional metrics related to overtime operations. The results indicate that the clinic typically will complete service for 14.2 of the 17 scheduled patients by noon. Even by 1300, the system completes all scheduled patients only 63.3% of the time. On average, the last patient scheduled for the morning session leaves at 1252 h, 292 min after the morning session starts, and experiences 50.4 min of waiting time. The next row of the table shows that if the system uses two residents instead of three, system performance deteriorates significantly. For example, the average number of patients seen by noon drops to 13.2. In addition, the last scheduled patient leaves the clinic at 1328 h, 328 min after the session started, and spends 87.0 min waiting within the system. On the other hand, when an examination room is added to the system, the impact is minimal, as shown in the row labeled “5 Rooms.”
Table 4also shows that changing the average teaching time produces results consistent with those shown in table 3. A 10% decrease in teaching time increases the average number of patient visits completed by noon from 14.2 to 14.5 and reduces the expected waiting time for the last patient on the schedule from 50.4 min to 45.4 min.
Performance metrics for the PP with no teaching mission, assuming the same mix and scheduling template for patient types, are shown in tables 5and 6. Comparison of the base cases in tables 3and 5shows that the average flow time in the PP setting is 129.1 min, compared with 76.2 min for the AMC. The average throughput is lower (2.7 vs. 3.5), the room utilization is lower (75.5%vs. 82.2%), and the utilization of the attending physician is higher (93.4%vs. 79.0%). Similarly, the utilization of the physician's assistant is higher (78.9%vs. 66.1%). The average waiting time is noticeably higher as well (83.9 vs. 30.0 min).
Table 5shows that the performance of the PP improves dramatically with the addition of a second physician. For example, average flow time is cut from 129.2 min to 99.2. However, such a solution is impractical because it drops the utilization of the attending physicians from 93.4% to 50.1%. Not surprisingly, reducing average time with the patient in the PP has a significant impact on performance metrics. Reducing time with the patient to 90, 80, 70, and 60% of its value in the base case cuts average flow time by 13.5, 27.4, 41.6, and 51.3 min, respectively. These changes would also cut average patient waiting time by 17.9, 37.9, 58.1, and 71.5 min, respectively.
Table 6shows performance metrics that parallel those shown in table 4. Again, based on a comparison of the base cases, the PP does not perform as well as the AMC. The average number of patient visits completed by noon is 10.3 versus 14.2. The average completion time of the last patient on the schedule is calculated to be 1429 versus 1252, and the average waiting time for that patient is 158.2 min versus 45.6 min in the AMC.
The primary goal of this study was to evaluate the impact of the teaching mission in an AMC on the performance metrics of clinical practice in an ambulatory pain center. To achieve this, we compared the simulated model of the AMC pain center and that of a PP clinic. A comparison of the base case settings for the two simulated clinics revealed that the performance of the AMC is superior to that of the PP clinic in terms of throughput, flow times, waiting times, and completion times. What we see here is a tradeoff. In the AMC, patients have less direct contact time with the attending physician but are effectively treated through the use of residents. However, the patient is involved in more process steps in the AMC. A type 1 patient's contact time alone with the resident averages 20.2 min; the time that the resident spends waiting for and interacting with the attending physician averages 8.3 min; and the patient's contact time with the attending physician averages 12.5 min, totaling 41.0 min. By comparison, the mean contact time between the patient and the attending physician in the PP clinic is 37.7 min. Thus, the AMC involves more resources, more activities, and more processing time per patient.
The additional processing time in the AMC is distributed between the attending physician and the residents. As tables 3and 5clearly show, with an identical schedule, the patient's waiting time is significantly shorter in the AMC than in the PP clinic. For the PP clinic to have comparable waiting times with the same number of patients, the attending physician must cut activity times by roughly 30%. Comparing the completion time of the final patient scheduled shows that the PP clinic would need to reduce the average processing time of the attending physician by more than 20% to match the average completion time of the AMC. We also note that the patient's waiting time is particularly sensitive to how much time the attending physician spends specifically on teaching the resident. For example, a 10% change in the average duration of step 6 in the AMC is approximately 40 s. A 40-s reduction in teaching time reduces the expected waiting time in the system by more than 2 min. This disproportionate impact occurs because teaching simultaneously occupies the three critical resources with the highest utilization levels in the system: the attending physician, the examination room, and the resident.
Considering throughput, the AMC averages 3.5 patients per hour versus 2.7 patients per hour in the PP clinic. The PP clinic would be able to match the throughput of the base case in the AMC by cutting the attending physician's average processing time by a little more than 20%. If we focus on completion time of the last job, we see that if the attending physician's activity time is cut by 30%, the average completion time is 289.9 min, a length of time comparable with the 292 min for the base case in the AMC. In this scenario, the utilization of the physician's assistant is 77.0% over the 289.9-min session. This compares to only 66.1% over the 291.9-min session in the base case of the AMC. This difference is explained by the extra time that the physician's assistant spends waiting to consult with the attending physician in the PP clinic setting.
Table 7shows how much higher the throughput is in the AMC compared with that of the PP for several scenarios. For the base cases, this value is (3.5 − 2.7)/2.7 = 29.6%. To explore the robustness of this result, we simulated additional scenarios. We simulated scenarios in which the time the attending physician spends with each patient, the teaching time, or both in the AMC are increased by 10%. We also simulated scenarios in which the time the attending physician spends with each patient in the PP is decreased by 10%, the last two patients on the schedule in the PP are dropped, or both. Table 7shows gains in throughput given all combinations of these changes. The scheme in which the performance gap is smallest includes a 10% reduction in the attending physician's processing time in the PP, the attending physician's processing time in the AMC is increased by 10%, and the teaching time is also increased by 10%. In this case, the throughput in the AMC is only 15.5% higher than that in the PP while both systems are serving patients. The gain in throughput in the most favorable scenario for the PP corresponds to an additional 2.6 patients per 4-h session. The revenue linked to this increase in throughput can be compared with the cost of the educational program to inform the tradeoff between the AMC's education mission and its impact on performance. A full cost analysis of the teaching mission is beyond the scope of this work but has been considered in more detail in previous works.18,19
All computer simulation models are theoretical instruments based on rules, generalizations, and assumptions. In addition, the natural experiment documented here was not explicitly created as a formal experiment. Consequently, our work is subject to several limitations. For one, we needed to estimate the distributions of the patient retrieval time used in the simulation and the time of the interaction between the attending physician and the physician's assistant for some type 3 cases. However, these times are both quite short and are common across the two settings. Consequently, moderate changes to these parameters have no impact on the performance gap between the two systems. In addition, the observers who gathered time data noted events, such as when a patient or caregiver entered or exited an examination room. They were not in a position to verify that all of the time labeled as step 5 in figure 1was actually “teaching time.” To address this potential concern, after the study data were gathered, we paid a different observer to shadow the clinic director for a day to explicitly measure teaching time. The results from this effort are that for six type 1 cases observed, teaching time ranged from 2 to 20 min, with a mean of 9.8 min. This compares with the range of 1–27 min with a mean of 8.3 min in the original (larger) data set. For the nine type 2 cases observed, this time ranged from 1 to 6 min, with a mean of 3.3 min. This compares with the range of 1–19 min, with a mean of 5.6 min in the original data. This sample supports the assumption that virtually all of the time between steps 4 and 6 can be labeled as “teaching time” and that there is no reason to doubt the appropriateness of the parameter values used in the simulation of this step.
In addition, the results presented here are based on institutional data from only one outpatient clinic in a very large AMC. Other sites within this AMC and at other AMCs may experience different characteristics and will require specific inputs to measure results in their institutions. Finally, although it is impossible to consider all conceivable scheduling rules, we did compare system performance under a variety of schedules. The AMC outperformed the PP in all settings considered. The smallest performance gap is obtained when using the schedule designed specifically for the PP. In this instance, the AMC outperformed the PP by 11.8%. The gap is smaller because the schedule includes fewer patients. It appears that given any schedule that results in high utilization of the attending physician, the AMC will perform better. Future research on scheduling rules can use or modify the simulations developed for this study when considering alternative settings.
Despite the potential limitations of simulation models, discrete-event simulation is a particularly useful tool for addressing the question at hand because it allows us to consider disparate systems under identical loads. It also exposes some results that are counterintuitive. For example, a 1-min increase in teaching time causes a roughly 3-min increase in average flow time in the AMC. This inflation occurs because different resources may be most critical at a given moment in time. Because this activity occupies three critical resources, it has a disproportionate impact on average flow times. These types of effects cannot be observed by using a static analysis or a simple queuing model because the interactions of resource availability cannot be captured using such approaches.
As a final thought, this study suggests that the objectives of greater efficiencies in the delivery of care and physician training must be considered simultaneously. Actions to improve the efficiency of the teaching mission may serve to increase the efficiency of both delivery of care and resident training. Future studies should consider explicitly the impacts of approaches such as taking some of the teaching activity “off-line” by preassigning patients to residents. Residents can review pertinent information and discuss the treatment plan with the attending physician before the patient encounter. Such approaches would alter the distributions of the activity times of several steps in the care delivery process.
The authors acknowledge the contribution of the residents, fellows, faculty, and staff of the Johns Hopkins Blaustein Pain Center. They also thank the team of data collectors: Konul Amrahova, B.A. (Research Assistant, Department of Original Investigations, Johns Hopkins University, Baltimore, Maryland), Heather Freeman, B.A. (Research Assistant, Towson University, Baltimore, Maryland), and Edmond Kim (Research Assistant, Carey Business School, Johns Hopkins University).