Validation of the Berlin Questionnaire and American Society of Anesthesiologists Checklist as Screening Tools for Obstructive Sleep Apnea in Surgical Patients

The study was conducted in the same patient population as described in the accompanying article.11Details of the inclusion and exclusion criteria, patient screening, sleep study and polysomnography scoring, and diagnosis and severity definition of OSA are described in that article. Approval from the Research Ethics Board of University Health Network and Mount Sinai Hospital (Toronto, Ontario, Canada) was obtained.

All patients who met the inclusion criteria and gave consent were screened by the three screening tools: the Berlin questionnaire, the ASA checklist, and the STOP questionnaire. Following a randomized order list, the STOP and Berlin questionnaires were clipped together and simultaneously administered to patients. Upon completion of the questionnaires and before scoring of the questionnaires, the patient was screened by one of the three research staff (two research anesthesiologists and a research assistant) with the ASA checklist. All patients who completed the questionnaires and the ASA checklist were invited to undergo an overnight in-laboratory polysomnographic study before surgery, regardless of their score on the questionnaires. The Berlin questionnaire and the ASA checklist were scored according to standard scoring criteria ( appendixes 1 and 2).

The reliability of the screening tools was checked before they were used to screen patients. The agreement and Cohen κ coefficient of test–retest were 96.3% (n = 54) and 0.9168 (confidence interval, 0.804–1.000), respectively, for the Berlin questionnaire and 96.4% (n = 55) and 0.923 (confidence interval, 0.818–1.000) for the STOP questionnaire. The Fleiss κ coefficient of the three research staff scoring the ASA checklist was 0.7460 (n = 29, P < 0.001).

If the apnea–hypopnea index (AHI) of a patient was greater than 30/h, the anesthesiologist and surgeon who were taking care of the patient were informed. The data regarding the perioperative complications of patients were obtained through chart review by a research anesthesiologist who was blinded to the results of the three questionnaires and polysomnography. The definition of postoperative complications was listed in appendix 3.

The details of the sample size estimation and data analysis are described in the accompanying article.11The test–retest agreement for the Berlin and the STOP questionnaire was analyzed with the Cohen κ coefficient. Interrater agreement among the three research staff for the ASA checklist was analyzed with the Fleiss κ coefficient. The Breslow-Day test was used to check whether there was a significant difference between the screening tools.

The analysis of the validation of the Berlin questionnaire and the ASA checklist, and the comparison of the three screening tools—the Berlin questionnaire, the ASA checklist, and the STOP questionnaire—were based on the 177 patients who underwent polysomnography and completed the three questionnaires. All 416 patients who gave consent were included in the postoperative complication analysis, with focus on the 211 patients who underwent polysomnography. The process of patient screening and the demographic data for the different groups of patients are described in the accompanying article.11 

In 2,467 screened patients who completed the three screening tools, 33% were classified as being at high risk of having OSA by the Berlin questionnaire, 27% by the ASA checklist, and 28% by the STOP questionnaire.

Table 1shows the demographic data of the patients regarding whether they were at high or low risk on the Berlin questionnaire, the ASA checklist, and the STOP questionnaire. Although the STOP questionnaire did not include any question regarding body mass index (BMI) and neck circumference, it was able to distinguish the patients with a significantly higher BMI and a larger neck circumference from patients with a lower BMI and a smaller neck circumference, similar to the Berlin questionnaire and the ASA checklist. Second, all three screening tools recognized the patients with significantly higher AHI. In addition, the Berlin and STOP questionnaires were able to identify patients with significantly lower minimum arterial oxygen saturation during overnight polysomnography. Third, besides hypertension, which is part of the STOP and Berlin questionnaires, there was a significantly higher prevalence of gastroesophageal reflux disease in patients classified as having a high risk of OSA by the STOP and Berlin questionnaires.

The scores of the three screening tools were evaluated versus  the AHI from overnight in-laboratory polysomnography. The predictive parameters of each screening tool for patients with mild, moderate, or severe OSA are shown in table 2. All three screening tools demonstrated a moderately high level of sensitivity for OSA screening. In terms of the specificity, in almost all situations that were checked, the 95% confidence intervals include 50%, which means that they were not significantly different from chance. When we conducted an overall comparison of the three screening tools, no significant difference was found in terms of the ability of the three screening tools to recognize patients with OSA, because the P  values were 0.378, 0.530, and 0.753 with AHI greater than 5, greater than 15, and greater than 30 as cutoffs in the Breslow-Day test for homogeneity of odds ratios.

The structural characteristics of the screening tools are summarized in table 3. Several features make the STOP questionnaire easiest to remember and to use among the three screening tools. These include a smaller number of items, a yes/no format of the question design, the simple mnemonic, and a straightforward scoring procedure.

Table 4briefly summarizes the demographic data and the postoperative complications of the 416 patients who consented to the study. There were no deaths or life-threatening complications in either group of patients. Compared with the patients who did not show up for polysomnography, the patients who underwent polysomnography had a significantly higher incidence of postoperative complications (22.8% vs.  14.6%; P = 0.034), mainly because of the increased incidence of severe desaturation (10.9% vs.  5.4%; P = 0.039). The patients who did not show up for polysomnography also had a significantly high rate of smokers (26.8% vs.  14.7%; P = 0.002).

Table 5summarizes the demographic data and postoperative complications in 211 patients who underwent polysomnography. The demographic data showed the same trend as in 177 patients.11Compared with patients with an AHI of 5 or less, the patients with an AHI greater than 5 were older and had a higher percentage of male patients. They also had a higher BMI, a larger neck circumference, and a higher prevalence of hypertension. Patients with an AHI greater than 5 had a significantly higher incidence of postoperative complications (table 5), as seen in the incidence of total complications (27.4% vs.  12.3%; P = 0.016), respiratory complications (22.6% vs.  9.2%; P = 0.021), and desaturation (20.6% vs.  9.2%; P = 0.044). As a result, more patients needed prolonged oxygen therapy (14.3% vs.  4.7%; P = 0.043). In terms of the incidence of postoperative complications at the different AHI cutoff values, there was no significant difference between patients with an AHI of 15 or less versus  patients with an AHI greater than 15, and patients with an AHI of 30 or less versus  patients with an AHI greater than 30.

When examining the frequency of postoperative complications from the perspective of the score of the screening tools (table 6), the patients ranked as high risk by the STOP questionnaire had a significantly higher incidence of respiratory complications (23.8% vs.  10.6%; P < 005), desaturation (22.2% vs.  9.4%; P < 0.05), and severe desaturation (15.1% vs.  4.7%; P < 0.05). The higher incidences of postoperative respiratory complications (25.7% vs.  9.9%; P < 0.05) and desaturation (21.4% vs.  8.5%; P < 0.05) were also found in the patients identified as having a high risk of OSA by the ASA checklist.

Table 7shows the odds ratios for the factors that are possibly related with the incidence of postoperative complications. In this patient population, gender, age older than 50 yr, BMI >35 kg/m², neck circumferences greater than 40 cm, hypertension, and gastroesophageal reflux disease were not significantly related to the incidence of postoperative complications. In terms of the screening tools, identification of high risk of having OSA by the STOP-Bang (an alternative scoring model of STOP questionnaire11) was significantly associated with the occurrence of postoperative complications. AHI greater than 5 was another significant factor for the occurrence of postoperative complications. When reviewing the subgroups with the different ranges of AHI, an AHI of 15–30 was the most significant risk factor for the postoperative complications.

This study has validated the use of the Berlin questionnaire and the ASA checklist as screening tools for OSA in surgical patients. Similar to the STOP questionnaire, both the Berlin questionnaire and the ASA checklist demonstrated a moderately high level of sensitivity, ranging from 65.6% to 87.2% for the different AHI cutoffs. The patients with OSA had an increased rate of postoperative complications, which was mainly due to the increased frequency of postoperative desaturation. Either having an AHI greater than 5 or being identified as being at high risk of having OSA by the STOP-Bang significantly increased the risk of postoperative complications.

Because of the high prevalence of OSA in surgical patients1,2and an increased awareness of OSA, anesthesiologists are dealing with an increasing number of patients with OSA.12The patients with undiagnosed OSA have increased perioperative morbidity and mortality.5,6,13Anesthesiologists require a practical and sensitive screening tool to identify patients at high risk of having OSA. Although many predictive models and questionnaires have been developed to identify patients at high risk of having OSA in the different patient populations,14–21none of them have been validated in surgical patients.

The Berlin questionnaire is a widely used screening tool for OSA. It was an outcome of the Conference on Sleep in Primary Care in April 1996 in Berlin, Germany. It includes 11 questions organized into the three categories, 5 questions related to snoring and the cessation of breathing in category 1, 4 questions related to daytime sleepiness in category 2, 1 question about high blood pressure, and 1 question regarding BMI in category 3. When two of three categories are classified as positive for a patient, the patient is rated as being at high risk of having OSA ( appendix 1).

The predictive performance of the Berlin questionnaire for OSA varies greatly among different patient populations. In primary care patients, the sensitivity and specificity were found to be 86% and 77%, respectively, at a cutoff of AHI greater than 5, and 54% and 97% at a cutoff of AHI greater than 15.7In a group of patients preselected by excluding all patients with any typical symptoms of OSA or any comorbidity that could significantly increase the risk of having OSA, a modified version of the Berlin questionnaire showed a sensitivity of 86% and a specificity of 96% at a cutoff of AHI greater than 15.22However, the sensitivity and specificity of the Berlin questionnaire were 62.5% and 53.8% with a cutoff of AHI of 10 or greater in 153 patients undergoing pulmonary rehabilitation. In patients referred to a sleep laboratory, the Berlin questionnaire again showed a very low predictive value. The sensitivity and specificity of the Berlin questionnaire were 68% and 49% at respiratory disturbance index greater than 5, 62% and 43% at respiratory disturbance index greater than 10, and 57% and 43% at respiratory disturbance index greater than 15.9 

Compared with the aforementioned studies, our results showed that the Berlin questionnaire had a moderately high level of sensitivity in surgical patients (68.9%) and a higher sensitivity for surgical patients with moderate and severe OSA (78.6–87.2%). However, the specificity is low and is not significant. This finding suggests that in surgical patients, the Berlin questionnaire is helpful in detecting the high risk of having OSA, especially if the OSA is moderate or severe.

The ASA Task Force on the Perioperative Management of Patients with Obstructive Sleep Apnea published a practice guideline in 2006.10These guidelines recommend the routine screening of surgical patients with a three-category checklist with 12 items for adults and 14 items for children ( appendix 2). The ASA checklist has never been validated in any group of patients. Our study is the first study that has evaluated the predictive values of the ASA checklist for OSA. Compared with the Berlin and STOP questionnaires, the ASA checklist demonstrated a similar level of sensitivity and specificity.

The STOP questionnaire was developed and validated in surgical patients.11There are four yes/no questions in the STOP questionnaire and eight yes/no items in the alternative scoring model STOP-Bang. The scoring is easy and straightforward. The STOP questionnaire performs with similar sensitivity and specificity compared with the Berlin questionnaire and the ASA checklist. The alternative scoring model STOP-Bang11demonstrated a high level of sensitivity (84–100%) and negative predictive value (61–100%), especially for moderate and severe OSA. If a patient is ranked as being at low risk of having OSA by the STOP-Bang, the patient will have a very low possibility of having moderate or severe OSA.

Most studies published on postoperative complications among OSA patients are focused on patients who underwent upper airway surgery.23–29Only a few studies have been published on postoperative complications in patients who underwent surgeries other than upper airway surgery.5,6,30,31The overall postoperative complication rate in OSA patients undergoing surgery other than upper airway surgery is increased, 39% versus  18% in the control group (P = 0.01). The rate of serious complications is 24%,6and the rate of respiratory complications is 32%.5Compared with the aforementioned studies, the overall rate of postoperative complications in our patients was lower (27.4% vs.  12.3%; P = 0.02). The most common complication was desaturation (20.6% vs.  9.2%; P = 0.04). There were no deaths or serious complications in our patients.

When individually checking the possible risk factors for postoperative complications, either being identified as being at high risk of having OSA by the STOP-Bang or having an AHI greater than 5 was associated with an increased occurrence of postoperative complications. When the subgroups with different AHI were further examined, patients with moderate OSA (AHI = 15–30) had a significantly increased risk for postoperative complications. However, the patients with severe OSA (AHI >30) did not show a similar increased risk for postoperative complications. Our ethics board required us to inform anesthesiologists if the patient's AHI was 30 or greater. In one of our study hospitals, we were required to admit all patients with an AHI of 30 or greater to the intensive care unit for postoperative observation for the first night after surgery. This requirement to monitor these patients in the intensive care unit may explain why AHI greater than 30 was not found to be a risk factor for postoperative complications in our study population.

Our data suggest that the patients identified as being at high risk of having OSA by the STOP questionnaire or by the ASA checklist had an increased postoperative complication rate. The finding may provide practical guidelines to anesthesiologists, but it must be confirmed with further study.

There are potential limitations with the study. Self-selection of patients may have been involved during the process of patient screening. The patients who had sleep symptoms might have selectively consented to overnight polysomnography. The patients who underwent polysomnography had a higher frequency of postoperative complications than the patients who did not show up for polysomnography, further supporting that there may have been self-selection from the perspective of patients. Additional potential limitations are discussed in the accompanying article.11 

In conclusion, the Berlin questionnaire and the ASA checklist have been validated in surgical patients as screening tools for OSA. Both demonstrated a moderately high level of sensitivity and a negative predictive value, as the STOP questionnaire did. The STOP questionnaire and the ASA checklist were also able to identify the patients susceptible to postoperative complications. Because of its easy-to-use format, the STOP questionnaire might be easier for patients to complete and more suitable in the busy preoperative clinics.

The authors thank all of the anesthesiologists at Toronto Western Hospital, Toronto General Hospital, and Mount Sinai Hospital (Toronto, Ontario, Canada).

Height _____ m Weight _____ kg Age_____ Male/Female

Please choose the correct response to each question.

1. Do you snore?

a. Yes

b. No

c. Don't know

If you snore: 

2. Your snoring is:

a. Slightly louder than breathing

b. As loud as talking

c. Louder than talking

d. Very loud—can be heard in adjacent rooms

3. How often do you snore?

a. Nearly every day

b. 3–4 times a week

c. 1–2 times a week

d. 1–2 times a month

e. Never or nearly never

4. Has your snoring ever bothered other people?

a. Yes

b. No

c. Don't know

5. Has anyone noticed that you quit breathing during your sleep?

a. Nearly every day

b. 3–4 times a week

c. 1–2 times a week

d. 1–2 times a month

e. Never or nearly never

6. How often do you feel tired or fatigued after your sleep?

a. Nearly every day

b. 3–4 times a week

c. 1–2 times a week

d. 1–2 times a month

e. Never or nearly never

7. During your waking time, do you feel tired, fatigued, or not up to par?

a. Nearly every day

b. 3–4 times a week

c. 1–2 times a week

d. 1–2 times a month

e. Never or nearly never

8. Have you ever nodded off or fallen asleep while driving a vehicle?

a. Yes

b. No

If yes: 

9. How often does this occur?

a. Nearly every day

b. 3–4 times a week

c. 1–2 times a week

d. 1–2 times a month

e. Never or nearly never

10. Do you have high blood pressure?

a. Yes

b. No

c. Don't know

Adapted from table 2in Netzer et al.  7 

The questionnaire consists of three categories related to the risk of having OSA.

Categories and scoring: 

Category 1:  items 1, 2, 3, 4, and 5

Item 1: If yes  is the response, assign 1 point.

Item 2: If c  or d  is the response, assign 1 point.

Item 3: If a  or b  is the response, assign 1 point.

Item 4: If a  is the response, assign 1 point.

Item 5: If a  or b  is the response, assign 2 points.

Category 1  is positive if the total score is 2 or more points.

Category 2:  items 6, 7, and 8 (item 9 should be noted separately)

Item 6: If a  or b  is the response, assign 1 point.

Item 7: If a  or b  is the response, assign 1 point.

Item 8: If a  is the response, assign 1 point.

Category 2  is positive if the total score is 2 or more points.

Category 3  is positive if the answer to item 10 is yes or  if the BMI of the patient is greater than 30 kg/m².

High risk of OSA:  two or more categories scored as positive

Low risk of OSA:  only one or no category scored as positive

Adapted from table 1in Gross et al.  10 

a. BMI ≥35 kg/m²

b. Neck circumference >43 cm/17 inches (men) or 40 cm/16 inches (women)

c. Craniofacial abnormalities affecting the airway

d. Anatomical nasal obstruction

e. Tonsils nearly touching or touching the midline

Two or more of the following are present (if patient lives alone or sleep is not observed by another person, then only one of the following need be present):

a. Snoring (loud enough to be heard through closed door)

b. Frequent snoring

c. Observed pauses in breathing during sleep

d. Awakens from sleep with choking sensation

e. Frequent arousals from sleep

One or more of the following are present:

a. Frequent somnolence or fatigue despite adequate “sleep”

b. Falls asleep easily in a nonstimulating environment (e.g. , watching TV, reading, riding in or driving a car) despite adequate “sleep”

c. [Parent or teacher comments that child appears sleepy during the day, is easily distracted, is overly aggressive, or has difficulty concentrating]*

d. [Child often difficult to arouse at usual awakening time]*

If two or more items in category 1 are positive, category 1 is positive.

If two or more items in category 2 are positive, category 2 is positive.

If one or more items in category 3 are positive, category 3 is positive.

High risk of OSA:  two or more categories scored as positive

Low risk of OSA:  only one or no category scored as positive

* Items in brackets refer to pediatric patients.