- Email: [email protected]
Measuring Patient Satisfaction — Be Prepared
Measuring Patient Satisfaction - Be Prepared Abstract: This article describes the approach used by the Queen Elizabeth II Health Sciences Centre to measure inpatient satisfaction at the recently merged institution. I t presents some of the issues that arose in the planning stages, including use of prior consent. I t is important to plan the methodology, the execution, the communication strategy and the follow-up before the process begins to ensure the results are credible, communicated and translated into quality improvement actions.
C
onsidering using a patient satisfaction survey? Plan ahead, and be prepared for issues that can arise in the process. The QEII Health Sciences Centre (QEII) is a 1,100-bed teaching hospital in Halifax, Nova Scotia that serves the people of the Halifax area as well as providing tertiary care for Nova Scotia and the other Atlantic Provinces. bY Pauline MacDonald
In 1995, the QEII was created from the merger of four adult care institutions. In 1997, with the opening of the new Halifax Infirmary building, many services were moved and consolidated. There was a general perception that the massive changes, the disruption and uncertainty had somehow affected patient care. One of the major measures of quality felt to be important at this time was patient satisfaction.
Healthcare Management Forum Gestion des soins de sand
Many services in each of the merged institutions had used patient questionnaires to get feedback, but the surveys varied so much in design and application that they did not provide a QEII wide perspective. One of the three Strategic Directions set by the QEII in 1998-99 was to Embrace Patient Focused Care. The executive decided to embark on a QEII-wide patient satisfaction survey process to provide feedback from the patients to assist in determining what improvements should be made to our customer service.
Why determine Patient Satisfaction? Patient satisfaction represents a complex mixture of perceived need, expectations, and experience of care. Without the patient’s viewpoint, an evaluation of service is incomplete and can be biased towards the provider’s perspective. Patient satisfaction is not a complete measure on its own, but supplements other indicators of quality of care. Patients have expectations for treatment outcomes and they can assess how well the care meets their needs. Their perceptions of the health benefit that they have derived from care are important. Although measures of satisfaction are subjective, they provide valuable insight into quality of care. The first question asked was “Why do we want the information?” For the QEII, both a comparison with other Canadian institutions and the imperative for change were important. Many people were confused by the merger; many felt betrayed and were loyal to one or another of the merged institutions. It was important to show how the QEII compared with other healthcare institutions across the country in order to give
43
the people of Nova Scotia and Atlantic Canada full confidence in the service the QEII was providing. The QEII Executive knew changes were still needed, but where? To what systems? What was important to staff? To physicians? To patients? The first step was getting patient feedback.
Choice of Vendor Having determined that the QEII was not prepared to design and administer its own survey, the first step was to look across Canada and determine what survey processes were in use. The search was narrowed down to three survey vendors - one Canadian and the other two American. The Canadian vendor offered inpatient surveying only at that time. Considering that Ambulatory care is the largest component of patient care, this restriction caused concern. One American vendor had a very good survey instrument, but when checking with current users it was suggested that the process and service were not acceptable. As well, no Canadian comparisons were available, but Canadian data comprised a small part of the general data bank. Some Toronto area hospitals were compiling their own data gathered from this vendor’s survey, but they were very disappointed with recent service. The other American vendor had very limited Canadian experience. There were basic criteria that had to be met:
+ The instrument had to be credible, valid, and reliable; + Results had to be linked to processes and improvement projects;
+ Comparison for benchmarking had to be within the Canadian Healthcare System. The decision was made to use the Conference Board of Canada as the vendor, using the “Measuring Up!” Survey tool. This decision was heavily influenced by the input received from physicians - they felt that the Conference Board process and comparisons would be more credible with the clinical groups, who would likely not accept American comparison. The Conference Board would not have an Emergency and Ambulatory Care survey tool for at least a year, so the QEII decided on a two-pronged patient satisfaction survey approach to deal with this dilemma. This paper considers the process for the inpatient population survey only.
Stratification The QEII wanted to survey the patient population as a whole and as subsets. However, the more detailed the stratification of survey sample, the smaller the numbers are and there is the danger that the numbers become too small to be meaningful. It also becomes more expensive as the total survey population must be larger. The QEII has about 30 services, but some, such as Emergency, have no inpatient component. Others are very small and could not feasibly form a separate stratum. These small services were grouped into broader medicine or surgery categories. In the end, 23 strata were selected. Tables were prepared based on physician service groupings. Only in a limited number of cases did units have more than one service that facilitated a unit level analysis. This was important to allow the area responsible for making changes to have unit-specific information. In addition to the cross-Canada
Healthcare Management Forum Gestion des soins de sant6
comparison, it was also important to look at the internal dynamics at the QEII, to compare processes and outcomes between service groups to see which were offering the most effective service and where services could learn from each other.
Consent Once a vendor was selected, other issues arose. The first was the requirement for prior consent. Many were concerned that patients should have been asked at admission or discharge for their permission to be contacted regarding a satisfaction survey. The need for prior consent from the legal, practical and ethical viewpoints was looked into, a review that held up the process for several weeks. The Conference Board of Canada provided us with a list of contacts at some of the other hospitals in the survey data base. Some of those contacted used a prior consent process, but most did not. Of those who did use prior consent, patients answered a mandatory question at admission/registration: “Do you want strict confidentiality?”If ‘no’, the field was marked in the registration system. If ‘yes’, a form was filled out to indicate if they preferred strict confidentiality or if contact for survey purposes was acceptable. One hospital excluded the ‘yes’ answers, psychiatry, abortions, and pediatrics, but now feel that they were too cautious and will probably include more next time.
As part of our Conference Board survey process, the patients received a letter signed by the hospital CEO asking them to participate. The hospital contracts with the Conference Board of Canada to perform a service for the hospital - legally, this makes them temporarily part of the hospital and thereforeconfidentialityis maintained. The survey is hospital-
44
driven and is used to improve hospital services. The Conference Board sends the introductory letter with the package, but do not initiate the letter. As well, no personal information regarding the nature of the patient’s admission is sent to the Board. The Conference Board does not share the information with others. One hospital stated that they did not get prior consent for the satisfaction survey, although they do for research surveys. They consider a satisfaction survey identical to following up on care given to a particular patient, and felt they could achieve the same purpose by asking patients’ opinions before they leave the floor, or later by letter. The introductory letter states that the patient is not obliged to participate, but the information is important to improve the care at the hospital. The patient has the option to refuse; their participation is their consent. The QEII Legal Counsel consulted with Legal Counsel at the Nova Scotia Department of Health. Their view was, that The Conference Board is acting as the hospital’s agent; they are not using the information for their own purposes, but doing it on behalf of the hospital and returning the results to the hospital for use in improving care. Therefore, no prior consent is required. That still left the ethical dilemmawas it fair to approach the patients without their permission? Although this issue was not resolved and many were still convinced prior consent was preferable, the legal and practical considerations swayed the decision in favour of not making the request. The QEII decided not to use prior consent for the following reasons:
+ Time to prepare and train the reception/admitting staff to record consent would cause delay of
several months. Because a new computer system had just been implemented, it would not be feasible to do this training just now. Use of prior consent could affect random sampling, as the patients would have “self-selected”to participate. It was not a legal requirement.
Process The Conference Board is experienced in survey administration and has a defined process to follow. In spite of that, a few problems were encountered. 1. The hospital was required to provide the Conference Board with a database containing the names of all live discharges over the most recent three-month period. This is important in avoiding upsetting a family by sending a survey to the address of patients who are deceased. However, other hospitals contacted cite this as a common problem. A three-month time span can mean that even though precautions are taken to delete names of the deceased, some die in the interim. One hospital said they even checked with vital statistics to ensure that people had not died since the last hospitalization, but it still happened that families of deceased patients received a survey. In the QEII’s case, we went back five months, as the new computer system implementation meant it was not easy to use very recent data.
2. In a very positive move, the QEII included a telephone number as a contact in case the recipient had any questions or concerns. The Patient Representative was the contact of choice, as this individual is accustomed to dealing with
Healthcare Management Forum Gestion des soins de sant6
patients and families who have issues to address. Concerns reported to the Patient Representatives included upset about deceased loved ones, perceived waste of money, or annoyance over receiving a second notice. The Conference Board sends out a reminder post card one week after the initial mailing, and a second survey two weeks after that, an effective means of increasing the rate of return. A tracking system deletes names from a second mailing once the completed surveys have been returned but does not have a way of tracking those that have been returned to the institution for other reasons (deceased, moved, do not wish to participate etc.). In retrospect, it would have been more efficient if clear instructions had been given to the QEII mailroom to direct all returned forms to the Planning and Quality Resources Department as quickly as possible. The Conference Board could then have been instructed to remove them from the list. This would not have completely solved the problem, however, as many were not returned by Canada Post until after the survey was terminated! 3. A possible approach might have been to have the Public Relations Department issue a public announcement that a survey was being done, but this might have created its own set of problems. The fact that the public would be aware and informed of some of the limitations may increase participation; on the other hand, telling the general public (and not just those surveyed) may increase the likelihood of objections to spending healthcare dollars for this purpose.
45
4. A process issue encountered by Patient Information Services (PIS) concerned the formatting of the data. The Conference Board did not at that time accept the formatting used by CIHI, so QEII staff had to reformat the information prior to sending it to the Conference Board; thus it was very time consuming to map all the codes for the twenty-three services profiled. The Conference Board now has the capacity to handle patient data with CIHI coding. A second problem was encountered when it was discovered that the way in which the QEII computerized system places mailing addressees was not compatible with the Conference Board requirements. Again, reformatting was time consuming and costly.
Results The QEII was very pleased with the Patient Satisfaction report. The response rate of 60% gave the QEII valuable and reliable information about patient satisfaction and their care experience.
With the presentation of the report, the work has just begun. The results from the Conference Board inpatient satisfaction report will be analyzed in light of the results of other reports: ambulatory patient satisfaction, employee satisfaction, physician satisfaction. Each service must now translate what our patients have told us into action so that we can make this an even better hospital to care for the people of Halifax, Nova Scotia and the Maritime provinces. A plan for distribution of the report to the users and education on how to make the best use of the information has been implemented. The intention is to inform as many people as possible about the results - if the results are not widely circulated, the lessons learned from the report will not have an impact on patient care. It is also important that the results are interpreted in a uniform manner, or the application will be inconsistent.
house, external agency) and the process for conducting the survey. Most important, it is how the results are communicated and translated into quality improvement actions that will determine the true value of knowing how your patients rate your care. The best approach is to plan the methodology, the execution, the communication and the follow-up before embarking on the process.
Pauline Macdonald is a Planning Consultant, Planning and Quality Resources Department, QEII Health Sciences Centre. She is presently on secondment as a Senior Policy Analyst, Acute Care Programs, the Nova Scotia Department of Health.
These broad measures
+ The QEII patients told us we are providing excellent patient care; + The QEII had the highest Total Satisfaction score in the Conference Board data base of 43 hospitals across Canada; + The QEII ranks high among its peer hospitals, with four fmt place rankings compared to the seven other teaching hospitals in the database; + Correlation scores indicated what aspects of care were most important to our patients (the main drivers of satisfaction); + The patients have provided clear indication of the areas that require improvement.
truly an imuortant measure of our success as a caring institution.
Conclusion The decision to include patient satisfaction as one of the measures of quality of care has ethical, practical and financial implications. The answer to “Why do we want the information?” will help guide the choice of method (in-
Healthcare Management Forum Gestion des soins de sant6
I
Our worldwide network of Healthcare consultants works with clients to measurably enhance their ability to build value, manage risk and improve their performance through:
Restructuring Planning and Implementation Client Relationship Management Facility and Business Planning E-Business and E-Health Health Reform and Evaluation Performance Benchmarking Analysis Information and Technology Management Operational, Clinical and Management Reviews Contact: Rik Canderton, National Healthcare Practice Leader tel: 41 6-941 -8209 / [email protected]
46
C
onsidering using a patient satisfaction survey? Plan ahead, and be prepared for issues that can arise in the process. The QEII Health Sciences Centre (QEII) is a 1,100-bed teaching hospital in Halifax, Nova Scotia that serves the people of the Halifax area as well as providing tertiary care for Nova Scotia and the other Atlantic Provinces. bY Pauline MacDonald
In 1995, the QEII was created from the merger of four adult care institutions. In 1997, with the opening of the new Halifax Infirmary building, many services were moved and consolidated. There was a general perception that the massive changes, the disruption and uncertainty had somehow affected patient care. One of the major measures of quality felt to be important at this time was patient satisfaction.
Healthcare Management Forum Gestion des soins de sand
Many services in each of the merged institutions had used patient questionnaires to get feedback, but the surveys varied so much in design and application that they did not provide a QEII wide perspective. One of the three Strategic Directions set by the QEII in 1998-99 was to Embrace Patient Focused Care. The executive decided to embark on a QEII-wide patient satisfaction survey process to provide feedback from the patients to assist in determining what improvements should be made to our customer service.
Why determine Patient Satisfaction? Patient satisfaction represents a complex mixture of perceived need, expectations, and experience of care. Without the patient’s viewpoint, an evaluation of service is incomplete and can be biased towards the provider’s perspective. Patient satisfaction is not a complete measure on its own, but supplements other indicators of quality of care. Patients have expectations for treatment outcomes and they can assess how well the care meets their needs. Their perceptions of the health benefit that they have derived from care are important. Although measures of satisfaction are subjective, they provide valuable insight into quality of care. The first question asked was “Why do we want the information?” For the QEII, both a comparison with other Canadian institutions and the imperative for change were important. Many people were confused by the merger; many felt betrayed and were loyal to one or another of the merged institutions. It was important to show how the QEII compared with other healthcare institutions across the country in order to give
43
the people of Nova Scotia and Atlantic Canada full confidence in the service the QEII was providing. The QEII Executive knew changes were still needed, but where? To what systems? What was important to staff? To physicians? To patients? The first step was getting patient feedback.
Choice of Vendor Having determined that the QEII was not prepared to design and administer its own survey, the first step was to look across Canada and determine what survey processes were in use. The search was narrowed down to three survey vendors - one Canadian and the other two American. The Canadian vendor offered inpatient surveying only at that time. Considering that Ambulatory care is the largest component of patient care, this restriction caused concern. One American vendor had a very good survey instrument, but when checking with current users it was suggested that the process and service were not acceptable. As well, no Canadian comparisons were available, but Canadian data comprised a small part of the general data bank. Some Toronto area hospitals were compiling their own data gathered from this vendor’s survey, but they were very disappointed with recent service. The other American vendor had very limited Canadian experience. There were basic criteria that had to be met:
+ The instrument had to be credible, valid, and reliable; + Results had to be linked to processes and improvement projects;
+ Comparison for benchmarking had to be within the Canadian Healthcare System. The decision was made to use the Conference Board of Canada as the vendor, using the “Measuring Up!” Survey tool. This decision was heavily influenced by the input received from physicians - they felt that the Conference Board process and comparisons would be more credible with the clinical groups, who would likely not accept American comparison. The Conference Board would not have an Emergency and Ambulatory Care survey tool for at least a year, so the QEII decided on a two-pronged patient satisfaction survey approach to deal with this dilemma. This paper considers the process for the inpatient population survey only.
Stratification The QEII wanted to survey the patient population as a whole and as subsets. However, the more detailed the stratification of survey sample, the smaller the numbers are and there is the danger that the numbers become too small to be meaningful. It also becomes more expensive as the total survey population must be larger. The QEII has about 30 services, but some, such as Emergency, have no inpatient component. Others are very small and could not feasibly form a separate stratum. These small services were grouped into broader medicine or surgery categories. In the end, 23 strata were selected. Tables were prepared based on physician service groupings. Only in a limited number of cases did units have more than one service that facilitated a unit level analysis. This was important to allow the area responsible for making changes to have unit-specific information. In addition to the cross-Canada
Healthcare Management Forum Gestion des soins de sant6
comparison, it was also important to look at the internal dynamics at the QEII, to compare processes and outcomes between service groups to see which were offering the most effective service and where services could learn from each other.
Consent Once a vendor was selected, other issues arose. The first was the requirement for prior consent. Many were concerned that patients should have been asked at admission or discharge for their permission to be contacted regarding a satisfaction survey. The need for prior consent from the legal, practical and ethical viewpoints was looked into, a review that held up the process for several weeks. The Conference Board of Canada provided us with a list of contacts at some of the other hospitals in the survey data base. Some of those contacted used a prior consent process, but most did not. Of those who did use prior consent, patients answered a mandatory question at admission/registration: “Do you want strict confidentiality?”If ‘no’, the field was marked in the registration system. If ‘yes’, a form was filled out to indicate if they preferred strict confidentiality or if contact for survey purposes was acceptable. One hospital excluded the ‘yes’ answers, psychiatry, abortions, and pediatrics, but now feel that they were too cautious and will probably include more next time.
As part of our Conference Board survey process, the patients received a letter signed by the hospital CEO asking them to participate. The hospital contracts with the Conference Board of Canada to perform a service for the hospital - legally, this makes them temporarily part of the hospital and thereforeconfidentialityis maintained. The survey is hospital-
44
driven and is used to improve hospital services. The Conference Board sends the introductory letter with the package, but do not initiate the letter. As well, no personal information regarding the nature of the patient’s admission is sent to the Board. The Conference Board does not share the information with others. One hospital stated that they did not get prior consent for the satisfaction survey, although they do for research surveys. They consider a satisfaction survey identical to following up on care given to a particular patient, and felt they could achieve the same purpose by asking patients’ opinions before they leave the floor, or later by letter. The introductory letter states that the patient is not obliged to participate, but the information is important to improve the care at the hospital. The patient has the option to refuse; their participation is their consent. The QEII Legal Counsel consulted with Legal Counsel at the Nova Scotia Department of Health. Their view was, that The Conference Board is acting as the hospital’s agent; they are not using the information for their own purposes, but doing it on behalf of the hospital and returning the results to the hospital for use in improving care. Therefore, no prior consent is required. That still left the ethical dilemmawas it fair to approach the patients without their permission? Although this issue was not resolved and many were still convinced prior consent was preferable, the legal and practical considerations swayed the decision in favour of not making the request. The QEII decided not to use prior consent for the following reasons:
+ Time to prepare and train the reception/admitting staff to record consent would cause delay of
several months. Because a new computer system had just been implemented, it would not be feasible to do this training just now. Use of prior consent could affect random sampling, as the patients would have “self-selected”to participate. It was not a legal requirement.
Process The Conference Board is experienced in survey administration and has a defined process to follow. In spite of that, a few problems were encountered. 1. The hospital was required to provide the Conference Board with a database containing the names of all live discharges over the most recent three-month period. This is important in avoiding upsetting a family by sending a survey to the address of patients who are deceased. However, other hospitals contacted cite this as a common problem. A three-month time span can mean that even though precautions are taken to delete names of the deceased, some die in the interim. One hospital said they even checked with vital statistics to ensure that people had not died since the last hospitalization, but it still happened that families of deceased patients received a survey. In the QEII’s case, we went back five months, as the new computer system implementation meant it was not easy to use very recent data.
2. In a very positive move, the QEII included a telephone number as a contact in case the recipient had any questions or concerns. The Patient Representative was the contact of choice, as this individual is accustomed to dealing with
Healthcare Management Forum Gestion des soins de sant6
patients and families who have issues to address. Concerns reported to the Patient Representatives included upset about deceased loved ones, perceived waste of money, or annoyance over receiving a second notice. The Conference Board sends out a reminder post card one week after the initial mailing, and a second survey two weeks after that, an effective means of increasing the rate of return. A tracking system deletes names from a second mailing once the completed surveys have been returned but does not have a way of tracking those that have been returned to the institution for other reasons (deceased, moved, do not wish to participate etc.). In retrospect, it would have been more efficient if clear instructions had been given to the QEII mailroom to direct all returned forms to the Planning and Quality Resources Department as quickly as possible. The Conference Board could then have been instructed to remove them from the list. This would not have completely solved the problem, however, as many were not returned by Canada Post until after the survey was terminated! 3. A possible approach might have been to have the Public Relations Department issue a public announcement that a survey was being done, but this might have created its own set of problems. The fact that the public would be aware and informed of some of the limitations may increase participation; on the other hand, telling the general public (and not just those surveyed) may increase the likelihood of objections to spending healthcare dollars for this purpose.
45
4. A process issue encountered by Patient Information Services (PIS) concerned the formatting of the data. The Conference Board did not at that time accept the formatting used by CIHI, so QEII staff had to reformat the information prior to sending it to the Conference Board; thus it was very time consuming to map all the codes for the twenty-three services profiled. The Conference Board now has the capacity to handle patient data with CIHI coding. A second problem was encountered when it was discovered that the way in which the QEII computerized system places mailing addressees was not compatible with the Conference Board requirements. Again, reformatting was time consuming and costly.
Results The QEII was very pleased with the Patient Satisfaction report. The response rate of 60% gave the QEII valuable and reliable information about patient satisfaction and their care experience.
With the presentation of the report, the work has just begun. The results from the Conference Board inpatient satisfaction report will be analyzed in light of the results of other reports: ambulatory patient satisfaction, employee satisfaction, physician satisfaction. Each service must now translate what our patients have told us into action so that we can make this an even better hospital to care for the people of Halifax, Nova Scotia and the Maritime provinces. A plan for distribution of the report to the users and education on how to make the best use of the information has been implemented. The intention is to inform as many people as possible about the results - if the results are not widely circulated, the lessons learned from the report will not have an impact on patient care. It is also important that the results are interpreted in a uniform manner, or the application will be inconsistent.
house, external agency) and the process for conducting the survey. Most important, it is how the results are communicated and translated into quality improvement actions that will determine the true value of knowing how your patients rate your care. The best approach is to plan the methodology, the execution, the communication and the follow-up before embarking on the process.
Pauline Macdonald is a Planning Consultant, Planning and Quality Resources Department, QEII Health Sciences Centre. She is presently on secondment as a Senior Policy Analyst, Acute Care Programs, the Nova Scotia Department of Health.
These broad measures
+ The QEII patients told us we are providing excellent patient care; + The QEII had the highest Total Satisfaction score in the Conference Board data base of 43 hospitals across Canada; + The QEII ranks high among its peer hospitals, with four fmt place rankings compared to the seven other teaching hospitals in the database; + Correlation scores indicated what aspects of care were most important to our patients (the main drivers of satisfaction); + The patients have provided clear indication of the areas that require improvement.
truly an imuortant measure of our success as a caring institution.
Conclusion The decision to include patient satisfaction as one of the measures of quality of care has ethical, practical and financial implications. The answer to “Why do we want the information?” will help guide the choice of method (in-
Healthcare Management Forum Gestion des soins de sant6
I
Our worldwide network of Healthcare consultants works with clients to measurably enhance their ability to build value, manage risk and improve their performance through:
Restructuring Planning and Implementation Client Relationship Management Facility and Business Planning E-Business and E-Health Health Reform and Evaluation Performance Benchmarking Analysis Information and Technology Management Operational, Clinical and Management Reviews Contact: Rik Canderton, National Healthcare Practice Leader tel: 41 6-941 -8209 / [email protected]
46