Use of patient satisfaction data in a continuous quality improvement program for endoscopic sinus surgery

Use of patient satisfaction data in a continuous quality improvement program for endoscopic sinus surgery CHIH-JAAN TAI, MD, MSC, CHIA-CHEN CHU, MSC, SHU-CHENG LIANG, MSC, TING-FU LIN, YAO-HAN TSAI, MD, and PA-CHUN WANG, MD, MSC, Taipei, Taiwan, Republic of China

OBJECTIVE: Continuous quality improvement (CQI) is an effort by health care providers to improve the quality of service by continuously exceeding patients’ expectations. Patient satisfaction is one of the measures of the quality of care. The aims of this study were to report the patients’ evaluation of endoscopic sinus surgery (ESS) and to explore the feasibility in using patient satisfaction data in the CQI program for ESS. METHODS: Eighty-three patients completed a validated patient satisfaction survey (PSS) 1 month after undergoing ESS. Logistic regression models were applied to determine the confounders of patient satisfaction. RESULTS: In general, 72% of patients were very satisfied with the services. Education level and milder disease correlated with higher overall satisfaction levels (P < 0.01). Anesthesia, the addition of nasal septal surgery, intranasal packing, and postoperative sinuscopy had significant impacts on patient satisfaction (P < 0.05).

From the Department of Otolaryngology, China Medical University Hospital and Department of Medicine, China Medical University (Dr Tai), Taichung, Taiwan; Department of Health, Republic of China (Misses Chu and Liang), Taipei, Taiwan; Department of Otolaryngology, Min-Sheng General Hospital (Drs Lin and Tsai), Taoyuan, Taiwan; and Department of Otolaryngology, Cathay General Hospital (Drs Huang and Wang), Taipei, Taiwan. Presented at the Annual Meeting of the American Academy of Otolaryngology–Head and Neck Surgery, Washington, DC, September 24-27, 2000. The research was sponsored by the Department of Health (DOH88-TD-1079), Taiwan, Republic of China. Reprint requests: Pa-Chun Wang, MD, MSc, Department of Otolaryngology, Cathay General Hospital, No. 280, Sec 4, Jen-Ai Road, 106, Taipei, Taiwan, Republic of China; e-mail, [email protected]. Copyright © 2003 by the American Academy of Otolaryngology–Head and Neck Surgery Foundation, Inc. 0194-5998/2003/$30.00 ⫹ 0 doi:10.1016/S0194-5998(02)00631-4 210

MD,

ZU-JIN HUANG,

MD,

CONCLUSION: ESS is a good technique to evaluate for implementing efforts in quality improvement. Confounding factors need to be adjusted before patient satisfaction data can be used in a CQI program. (Otolaryngol Head Neck Surg 2003;129:210-6.)

Q uality of care is being increasingly emphasized in a competitive managed-care environment.1,2 Continuous quality improvement (CQI) can be described as the continual attempt to furnish care that meets or exceeds patient expectations.3,4 The scopes of quality of care include accessibility, appropriateness, clinical outcomes, and patient satisfaction with care. Patient satisfaction has become one of the most important features in modern health care quality.5-7 Patient satisfaction can be measured by validated instruments.5-7 However, concerns about the validity of patient satisfaction data have limited its application and usefulness in quality improvement programs. Possible confounding effects derived from patient characteristics, providers, and various medical processes need to be adjusted to make a fair comparison.8 A standardized surgical procedure, such as endoscopic sinus surgery (ESS), is a good candidate for implementing CQI efforts. The aims of this study were to report the patients’ evaluation of the ESS they received and to investigate the factors that may confound the level of satisfaction. Issues regarding the use of patient satisfaction data in a CQI program for ESS are also addressed. MATERIALS AND METHODS The study was conducted in a prospective manner at a tertiary referral otolaryngology department. Eighty-three consecutive patients with chronic sinusitis were enrolled in the study, which was carried out between July and December 1998. The diagnosis of chronic sinusitis was established in compliance with the 1997 American Academy of Otolaryngology–Head and Neck Surgery Rhinosinusitis Task Force definition.9 Patients were

Otolaryngology– Head and Neck Surgery Volume 129 Number 3

Table 1. Features to evaluate in the patient satisfaction survey* Overall evaluation Discomfort during operation Postoperative discomfort Professional skill of caregiver Personal manner of hospital staff Explanation of condition by caregiver Time spent for service Office visit waiting time Administrative efficiency *Response categories: excellent, very good, good, fair, poor.

all evaluated by sinus computed tomography scan before ESS and were asked to answer a validated patient satisfaction survey (PSS) 1 month after surgery. Modified from the model of Rubin et al,10 the PSS was designed specifically to measure the satisfaction level of surgical patients by surveying 9 features of service. Components of the PSS are shown in Table 1. Patients rated each feature with a 5-point Likert-type scale of 1 ⫽ excellent, 2 ⫽ very good, 3 ⫽ good, 4 ⫽ fair, and 5 ⫽ poor. Higher satisfaction was determined as a lower score. The PSS is in traditional Chinese format and was validated before use. Patient demographics (age, gender, education) and characteristics (disease severity, underlying comorbidities, past history of nasal surgery, and predisposing conditions such as asthma or allergy) were collected on entry into this study. Operationrelated information such as provider, type of anesthesia, additional septal or turbinate surgery, types of intranasal packing, blood loss, use of prophylactic antibiotics, operating room use time, and operation time and postoperative information such as adverse events, number of sinuscopic examinations, and date of removal of the packing were retrospectively collected from medical records. The Harvard staging system proposed by Gliklich and Metson11 was used to document the extent of disease. Blood loss of less than 50 mL during surgery was defined as mild; 51 to 200 mL as moderate, and more than 200 mL as excessive. A stepwise multilevel ordinal logistic regression model was applied, with levels of satisfaction serving as dependent variables and patients’ data (characteristics and demographic and clinical information) as independent variables. Microsoft

TAI et al 211

Table 2. Demographic information (N ⫽ 83) Age (yr) Median SD Range Gender, n (%) Male Female Education (yr) ⬍1 1–6 7–9 10–12 13–16 ⬎16 Incomplete information

37.4 13.3 18–71 47 (56.6) 36 (43.4) 1 (1.3) 12 (15.4) 5 (6.4) 28 (35.9) 30 (38.5) 2 (2.6) 5 (6.4)

Access 7.0 software was used to design the research database, and SAS statistical software (SAS Institute, Cary, NC) was used to conduct the analyses. The objective of statistical analyses was to identify variables that were predictive of patient satisfaction at a significance level of 0.05. RESULTS Demographic and Operation-Related Data The demographic information for these patient cohorts are presented in Table 2. The patients ranged in age from 18 to 71 years (average age, 37.4 years; 47 [56.6%] male and 36 [43.4%] female). Patients’ clinical, operative. and postoperative information is given in Table 3. Thirty-seven patients (44.6%) received ESS under local anesthesia, whereas 46 patients (55.4%) were under general anesthesia. There were no differences between the distributions of gender and type of anesthesia. There were 4 adverse events documented in the medical records for this patient population, including 2 patients with excessive bleeding in the postoperative period, 1 patient with an orbital hematoma, and the other patient who experienced suspicious cerebrospinal fluid rhinorrhea. Patient Satisfaction Data The results of the PSS showed that most of the patients were quite satisfied with the services, the distribution of satisfaction levels being skewed to the left (Fig 1). Generally speaking, 72.3% of patients rated the “overall” services as excellent or very good. Satisfaction with the “personal man-

Otolaryngology– Head and Neck Surgery September 2003

212 TAI et al

Table 3. Clinical, operative, and postoperative information (N ⫽ 83) Staging, n (%) I II III IV Anesthesia, n (%) Local General Type of operation, n (%) Endoscopic surgery only Combined with framework surgery Combined with external approach Blood loss, n (%) Mild Moderate Excessive Nasal packing, n (%) No packing Surgicel only Either Vaseline, Iodoform, or Merocel Adverse event, n (%) Prophylaxis antibiotics, n (%) Operating room use time (min) Operation time (min) Hospital length of stay No. of visits in 6-mo period after surgery No. of sinuscopy procedures in 6-mo period after surgery

36 (44.5) 29 (35.8) 13 (16) 3 (3.7) 37 (44.6) 46 (55.4) 47 (56.6) 30 (36.2) 6 (7.2)

52 (62.7) 19 (22.9) 12 (14.4) 6 (7.2) 36 (43.4) 41 (49.4) 4 (4.8) 11 (13.3) 107.1 ⫾ 55.6 88.9 ⫾ 48.2 3.9 ⫾ 1.1 days 9.6 ⫾ 4.0 times 3.7 ⫾ 3.1 times

ner” of caregivers and “explanation of condition” from physicians were both over 90%. Satisfaction with physicians’ “professional skill” was over 80%. Relatively, “postoperative discomfort” and “office-visit waiting time” were poorly rated, with 18.1% and 40.9% fair or poor ratings ranked, respectively. “Discomfort during operation,” “time spent for service,” and “administrative efficiency” were rated as fair by 6%, 8.4%, and 8.4% of the patients respectively (Table 4). Satisfaction Confounders All independent variables (patients’ demographic information, characteristics, clinical information, operation-related data) were entered into stepwise multiple logistic regression models to determine whether

they were predictive of patient satisfaction levels. Significant confounders are shown in Tables 5 and 6. Positive ␤ value indicated higher odds of being dissatisfied with services. The results of the analysis showed that “overall” satisfaction with the ESS they received was significantly related to both disease severity and their level of education. Patients with a higher level of education were more satisfied with the services (␤ ⫽ ⫺0.65, P ⫽ 0.009). Level of satisfaction negatively correlated with the extent of sinusitis (␤ ⫽ 0.5, P ⫽ 0.01). Satisfaction with “discomfort during operation” was significantly related to age, local anesthesia, additional septal/turbinate surgery, operation room use time, and operation time. Tolerance to operative pain slightly increased with age (␤ ⫽ ⫺0.04, P ⫽ 0.03). Local anesthesia proved to be capable of inducing much discomfort during the surgery (␤ ⫽ 22.02, P ⫽ 0.005). Longer operation time induced more discomfort (␤ ⫽ 0.06, P ⫽ 0.0042). Satisfaction with “postoperative discomfort” was significantly related to age, local anesthesia, additional septal/turbinate surgery, and high-pressure packing (Vaseline gauze, Merocel, Iodoform). Tolerance to postoperative discomfort increased slightly with age (␤ ⫽ ⫺0.05, P ⫽ 0.01). Longer operation time correlated with more postoperative discomfort (␤ ⫽ 0.04, P ⫽ 0.01). Patients suffered more from postoperative discomfort due to the high pressure from intranasal packing (␤ ⫽ 9.72, P ⫽ 0.008). The effect of pressure from packing was strong. Satisfaction with “professional skill” was significantly related to patient’s level of education, additional septal/turbinate surgery, and number of postoperative sinuscopic examinations. Patients with higher educational level better appreciated the professionalism of physicians (␤ ⫽ ⫺0.67, P ⫽ 0.01). Patients who received frequent postoperative sinuscopic procedures were more satisfied with the physician’s professional skills (␤ ⫽ ⫺0.32, P ⫽ 0.02). Satisfaction with “personal manner” was significantly related to the frequency of postoperative sinuscopic procedures. Patients’ satisfaction level with “personal manner” (␤ ⫽ ⫺0.32, P ⫽ 0.04) slightly increases with the number of. postoperative endoscopic examinations and treatments.

Otolaryngology– Head and Neck Surgery Volume 129 Number 3

TAI et al 213

Fig 1. Results of patient satisfaction survey.

Table 4. Distributions of PSS feature-specific satisfaction levels Patients’ rating, n (%) Rating item

Excellent

Very good

Good

Fair

Poor

Overall Discomfort during operation Post operative discomfort Professional skill Personal manner Explanation of condition Time spent for service Office visit waiting time Administrative efficiency

36 (43.4) 25 (30.1) 18 (21.7) 47 (56.6) 51 (61.4) 50 (60.2) 32 (38.6) 11 (13.3) 12 (14.5)

24 (28.9) 32 (38.6) 20 (24.1) 20 (24.1) 27 (32.5) 27 (32.5) 27 (32.5) 14 (16.9) 28 (33.7)

23 (27.7) 21 (25.3) 30 (36.1) 14 (16.9) 5 (6.0) 6 (7.2) 16 (19.3) 24 (28.9) 27 (32.5)

0 (0.0) 5 (6.0) 12 (14.5) 0 (0.0) 0 (0.0) 0 (0.0) 7 (8.4) 27 (32.5) 7 (8.4)

0 (0.0) 0 (0.0) 3 (3.6) 0 (0.0) 0 (0.0) 0 (0.0) 0 (0.0) 7 (8.4) 0 (0.0)

*Row may not total 100% due to missing data.

Satisfaction with “explanation of condition” from physicians was significantly related to additional septal/turbinate surgery and pressure related to intranasal packing. Patients with additional framework surgery and pressure from packing were more satisfied with perioperative counseling (␤ ⫽ ⫺5.17, P ⫽ 0.04). Satisfaction with “time spent for service” was significantly related to local anesthesia, additional septal/turbinate surgery, occurrence of adverse events, intranasal packing, frequency of postoperative sinuscopy, prophylactic antibiotics, and blood loss. Patients who received non-

pressure intranasal packing such as Surgicel (␤ ⫽ ⫺9.13, P ⫽ 0.005) and frequent postoperative endoscopic treatments (␤ ⫽ ⫺0.39, P ⫽ 0.005) were more satisfied with the “time spent for service.” Service time spent on patients who had experienced adverse events was highly appreciated (␤ ⫽ ⫺4.47, P ⬍ 0.01). Patients who were operated on under local anesthesia (␤ ⫽ 11.56, P ⬍ 0.01), received preoperative prophylactic medication, and experienced significant blood loss (␤ ⫽ 0.02, P ⬍ 0.05) during surgery were not satisfied with the service time they received.

Otolaryngology– Head and Neck Surgery September 2003

214 TAI et al

Table 5. Patient satisfaction confounders I ␤ for Independent Variables Dependent variable

Stage

Education

Overall Discomfort during operation Postoperative discomfort Professional skill Personal manner Explanation of condition Time spent for service Office visit waiting time Administrative efficiency

0.5†

⫺0.64†

Age

⫺0.04* ⫺0.05†

⫺0.67†

LA

22.02‡ 7.96*

11.56† 8.36* 7.17*

0.5†

ESFS

OR use time

OP time

0.07‡

0.06‡

⫺7.1‡ ⫺3.22* ⫺3.1* ⫺5.1* ⫺3.06* ⫺3.93‡

0.03*

LA, Local anesthesia; ESFS, endoscopic surgery plus framework surgery; OR, operation room; OP, operation. *P ⱕ 0.05. †P ⱕ 0.01. ‡P ⱕ 0.005.

Table 6. Patient satisfaction confounder II ␤ for independent variables Dependent variable

Overall Discomfort during operation Postoperative discomfort Professional skill Personal manner Explanation of condition Time spent for service Office visit waiting time Administrative efficiency

HP packing

9.72† ⫺5.17* ⫺3.42‡ ⫺3.12‡ ⫺2.2*

Sinuscopy number

PA

Blood loss

Adverse event

Surgicel packing

0.02*

⫺4.47†

⫺9.13‡

⫺0.32* ⫺0.32* ⫺0.39‡ ⫺0.23*

5.03* 5.09* 4.33*

HP, high-pressured; PA, prophylactic antibiotics. *P ⱕ 0.05. †P ⱕ 0.01. ‡P ⱕ 0.005.

Satisfaction with “office visit waiting time” was significantly related to local anesthesia, additional septal/turbinate surgery, operating room use time, high-pressure packing, and prophylactic antibiotics. Patients who received additional framework surgery (␤ ⫽ ⫺3.93, P ⫽ 0.005) and high-pressure packing (␤ ⫽ ⫺3.12, P ⫽ 0.0059) were less dissatisfied with the office visit waiting time. Satisfaction with “administrative efficiency” was significantly related to extent of disease, local anesthesia, high-pressure packing, frequency of sinuscopy, and prophylactic antibiotics. Patients with more severe disease were more likely to complain about the efficiency of administration (␤ ⫽ 0.5, P ⫽ 0.008).

Patients who received local anesthesia (␤ ⫽ 7.17, P ⫽ 0.03) or had used prophylactic medication were less satisfied with the efficiency of administration. DISCUSSION Quality of care has been vehemently emphasized in a patient-centered modern health care environment. Patient satisfaction, an evaluation of services from the consumers’ point of view, has been widely used in quality management of health care. Researches have focused extensively on issues such as validity of data, risk adjustment, and fairness of comparison when using patient satisfaction data in continuous quality improvement

Otolaryngology– Head and Neck Surgery Volume 129 Number 3

programs.3,10,12-14 Standardized medical-surgical procedures, such as ESS, are good candidates for implementing CQI efforts. However, limited studies13 in the past have published data regarding the patients’ satisfaction with ESS. This study was an attempt to investigate the feasibility of using satisfaction data in a CQI program specifically for ESS. Investigators have suggested a multidimensional approach so as to detect different features of patient satisfaction.10 Modified from Rubin and Ware’s model, a 5-level Likert-type scale was chosen to increase response sensitivity. The PSS covered 9 important components of services and was customized specifically to be used by surgical patients.1 The results of our study demonstrated that both disease severity (computed tomography scan stage) and education levels confounded the overall satisfaction level. Regarding why patients with severe disease or lower education level tended to be dissatisfied, those patients may underestimate or even misunderstand the complexity of sinusitis and the difficulty of treatment. This hypothesis also explains why highly educated patients showed more appreciation of physicians’ professional skill and performance. Milder disease and higher education have previously been shown to be significantly associated with higher satisfaction to health care services.8,15 Our study has confirmed that the phenomena are also applicable for ESS patients. Local anesthesia had strong and consistent negative impacts on “discomfort during operation” and many other features of patient satisfaction, proving that the survey was a valid measure. The fact that local anesthesia is a significantly negative confounder to discomfort during and after the operation may explain the trend in preference shifting from local to general anesthesia. Local anesthesia also strongly correlated with time spent for service, office waiting time, and administrative efficiency, probably due to the relatively milder disease severity of these patients. Experience has told us that patients with milder disease tend to be more sensitive to the time cost spent for their own health care. Age is a positive but minor confounder to discomfort during and after the operation. Older patients seemed to tolerate pain better

TAI et al 215

than younger patients. The fact that high-pressure packing (eg, Vaseline gauze, Merocel, Iodoform) can result in postoperative discomfort is quite straightforward; our data also showed that time spent to clean up debris meticulously was highly appreciated by patients with Surgicel packing. The frequency of postoperative sinuscopy had positive influences on some features of patient satisfaction, including professional skill, personal manner, time spent for service, and administrative efficiency. The role of postoperative sinuscopy has been controversial in the ESS literature,16 and physicians may assume that frequent postoperative sinuscopy could be bothersome to patients. To the contrary, our data have shown that patients seemed to value the postoperative sinuscopy, even though the effect appeared to be minor. Our data also showed that asking patients to comply with a prophylactic medication schedule was not appreciated by increasing their time cost. There have been doubts about the fairness of comparisons between different sets of satisfaction data.17 Previous studies have claimed that patients ratings were biased by general attitude, lifestyle, and perceptions of health.18 The effect of education to overall satisfaction level in our study was an example. Validity of satisfaction level was also criticized because of their association with demographic characteristics, expectations, preference, and priorities of patients.13,19 Our studies have revealed the effects of many confounding factors that were previously unknown to many regular ESS performers. Confounding effects of different health care providers, databases, and time frames should also be taken into consideration15 for the future use of this data set. Findings in our study supported the notion that comparison of satisfaction level without risk adjustment is baseless. Feature-specific PSS is an efficient measure to provide worthwhile patient satisfaction information for a CQI program. Results from this study will serve as baseline data in future CQI programs for ESS. Physicians and health care organizations can monitor their performance and services by tracking serial PSS data to continuously improve their quality in caring for ESS patients.

Otolaryngology– Head and Neck Surgery September 2003

216 TAI et al

CONCLUSION Standardized surgical procedure such as ESS is a good model for implementing the CQI program. The efforts to continuously improve quality of care can be documented with patient satisfaction measures. However, this study has shown that patient characteristics and process variables had various impacts on different features of patient satisfaction. To use patient satisfaction information in a CQI program for ESS, a feature-specific approach is necessary for the best discretion on different dimensions of quality of care and proper statistical adjustment for patient characteristics and process variables is mandatory in making fair data comparison.

8. 9. 10. 11. 12. 13.

REFERENCES

1. Ware JE Jr, Hays RD. Methods for measuring patient satisfaction with specific medical encounters. Med Care 1988;26:393-402. 2. Murray JP: A follow-up comparison of patient satisfaction among prepaid and fee-for-service patients. J Fam Pract 1988;26:576-81. 3. Berwick DM. Continuous improvement as an ideal health care. N Engl J Med 1989;320:53-6. 4. Berwick DM, Godfrey AB, Roessner J. Curing health care. San Francisco, CA: Jossey-Bass; 1990. 5. Brook RH, Lohr KN. Efficacy, effectiveness, variations, and quality. Med Care 1985;23:710-22. 6. Ware JE Jr, Davies-Avery A, Stewart AL. The measurement and meaning of patient satisfaction. Health Med Care Serv Rev 1978;1:2-15. 7. Ware JE Jr, Synder MK, Wright WR. Development and validation of scales to measure patient satisfaction with

14.

15. 16. 17. 18. 19.

health care services: volume of a final report. Part A: review of the literature, overview of methods, and results regarding construction of scales. PB-288-329. Springfield, VA: National Technical Information Service; 1976. Young GJ, Meterko M, Desai KR. Patient satisfaction with hospital care: effects of demographic and institutional characteristics. Med Care 2000;38:325-34. Lanza DC, Kennedy DW, et al. Adult rhinosinusitis defined. Report of the Rhinosinusitis Task Force. Otolaryngol Head Neck Surg 1997;117:S1-7. Rubin HR, Gandek B, Rogers WH, et al. Patients’ rating of outpatient visits in different practice setting. JAMA 1993;270:835-40. Gliklih RE, Metson RB. A comparison of sinus computed tomography (CT) staging systems for outcomes research. Am J Rhino 1994;8:291-7. Holland MS, Counte MA, Hinrichs BG. Determinants of patient satisfaction with outpatient surgery. Qual Mgmt Health Care 1995;4:82-90. Piccirillo JF. The use of patient satisfaction data to assess the impact of continuous quality improvement efforts. Ach Otolaryngol Head Neck Surg 1996;122:1045-8. Ware JE Jr, Davies AR, Rubin HR. Patients’ assessments of their care. In: The quality of medical care: Information for consumers. Washington, DC: US Congress, Office of Technology Assessment; 1988;231-47. Sixma HJ, Preeuwenberg P, Van Der Pasch M. Patient satisfaction with the general practitioner: a two-level analysis. Med Care 1998;36:212-29. Fernandes SV. Postoperative care in functional endoscopic sinus surgery Laryngoscope 1999;109:945-8. Woodbury D, Tracey D, McNight E. Does considering severity of illness improve interpretation of patient satisfaction data? Health Care Qual 1998;20:33-40. Linn LS, Greenfield S. Patient suffering and patient satisfaction among the chronically ill. Med Care 1982;20: 425-31. Davies AR, Ware JE Jr. Involving consumers in quality of care assessment. Health Aff 1988:7:33-48.