- Email: [email protected]
Measuring Patient and Family Experiences of Health Care for Children
Measuring Patient and Family Experiences of Health Care for Children John Patrick T. Co, MD, MPH; Scot B. Sternberg, MS; Charles J. Homer, MD, MPH From the Department of Pediatrics and Center for Child and Adolescent Health Policy, Massachusetts General Hospital, Boston, Mass (Dr Co); National Initiative for Children’s Healthcare Quality, Boston, Mass (Mr Sternberg and Dr Homer); and Department of Pediatrics, Harvard Medical School, Boston, Mass (Dr Co and Dr Homer) The views expressed in this report are those of the authors and do not necessarily represent those of the Department of Health and Human Services, Agency for Healthcare Research and Quality, or Centers for Medicare & Medicaid Services. Please see Acknowledgments section for conflicts of interest information. Publication of this article was supported by the US Department of Health and Human Services or the Agency for Healthcare Research and Quality. Address correspondence to John Patrick T. Co, MD, MPH, MGH Center for Child and Adolescent Health Policy, 50 Staniford Street, Suite 901, Boston, Massachusetts 02114 (e-mail: [email protected]). Received for publication March 9, 2010; accepted January 24, 2011.
ABSTRACT BACKGROUND: The Institute of Medicine considers patient
Survey (NRC Picker). We identified the need for matching patients with providers and groups as a barrier for widespread use of the CAHPS pediatric clinician & group instrument. CONCLUSIONS: We recommended to the National Advisory Council for Healthcare Research and Quality Subcommittee on Children’s Healthcare Quality Measures for Medicaid and Child Health Insurance Programs (SNAC) the CAHPS Child Medicaid 4.0 and pediatric Clinician & Group Survey for inclusion in the initial recommended list of core measures for voluntary use by Medicaid and CHIP. The Clinician and Group Survey was not included in the list posted for public comment due to concerns at that time (December 2009) about feasibility. We also recommended that development of a child version of the CAHPS behavioral and mental health survey now used in the adult population and of a pediatric hospital CAHPS measure be considered high priorities for development in the next phase of Children’s Health Insurance Program Reauthorization Act measurement activity. This phase should also explore methods to increase response rates and lower costs of obtaining consumer feedback.
centeredness a core dimension of quality. Several patient/family surveys exist to assess pediatric health care. The Children’s Health Insurance Program Reauthorization Act mandates strengthening quality measurement for children, including for patient/family experience of care. OBJECTIVES: The aim of this study was to determine what instruments exist for measuring patient/family experience of pediatric health care and which should be included in the core measurement set for assessing Medicaid and the Children’s Health Insurance Program (CHIP) programs; to identify gaps in measurement; and to provide recommendations for measure development. METHODS: We developed a conceptual framework for measuring patient/family experience of care. We conducted a review of national measure clearinghouses and of the literature to assess validity, reliability, and feasibility of existing measures, and how these measures address the conceptual framework. RESULTS: We found valid and reliable instruments for measuring patient/family experience of care include the Consumer Assessment of Healthcare Providers and Systems (CAHPS) instruments, Promoting Healthy Development Survey (PHDS), Young Adult Health Care Survey (YAHCS), and the National Research Corporation Picker Pediatric Inpatient
KEYWORDS: patient centeredness; patient experience; quality of care; surveys ACADEMIC PEDIATRICS 2011;11:S59–S67
ON FEBRUARY 4, 2009, the Children’s Health Insurance Program Reauthorization Act (Public Law 111-3) was signed into law by President Obama, reauthorizing the Children’s Health Insurance Program (CHIP). Included in the law was a mandate to strengthen quality of care and health outcomes for children enrolled in Medicaid and CHIP programs, and specific language directing the Secretary of Health and Human Services to identify a recommended core set of child health quality measures for voluntary use by state Medicaid and CHIP programs by January 1, 2010. These measures, to include assessment of patient/family experience of care, should provide information on the quality of health care and coverage of children across the age and health spectrum and ACADEMIC PEDIATRICS Copyright ª 2011 by Academic Pediatric Association
provide helpful information for improving the quality of care for children. The Institute of Medicine considers patient centeredness a core dimension of quality, one that should be included in assessing the quality of health care delivered in the United States and that can only be assessed through patient reports of care.1 Patient-centered heath care includes establishing a partnership between providers, patients, and their families, as well as the patient experience itself. Improvements in health care design and delivery should include making the system more patient centered.2 Patient-centered approaches to care are associated with improvements in care, including for chronic care management.3–6 In particular, effective communication is associated with improved patient satisfaction and adherence.7,8
S59
Volume 11, Number 3S May–June 2011
S60
CO ET AL
CONCEPTUAL FRAMEWORK FOR MEASURING PATIENT/ FAMILY EXPERIENCE ASPECTS OF CARE THAT CAN BE REPORTED AND MEASURED The development of the initial versions of the Consumer Assessment of Healthcare Providers and Systems (CAHPS) instruments (including the child survey) for assessing health plans provided a critical foundation for measuring patient/ family experience.9–11 Focus groups and extensive cognitive testing revealed that domains such as access to care, timeliness of care, communication, office staff, and health plan customer service could validly and reliably be reported by adult respondents. Further work, including literature review and interviews with experts in pediatric medical care and health plan satisfaction, determined that the content of the child survey should differ from the content of the adult survey.12 Some additional areas were identified, including the child’s interactions and communication with health care professionals; preventive care; and care for the child’s growth, development, or behavior. REPORTING ON CHILDREN’S HEALTH CARE Survey data about children’s health care is usually collected from parents rather than directly from children, for both logistic and methodological reasons.13,14 Issues such as obtaining informed consent, contacting children, and interviewing them are substantial challenges for obtaining survey responses in the pediatric population.15–17 For example, children aged younger than 12 years tend to respond positively to survey questions regardless of their content.18 The limited and changing cognitive ability of children limits the complexity of item wording. These types of issues have prompted most to assess children’s health care by using parent report. Measurement of patient/family experience with health care is a particular challenge in the adolescent population.19 Although usually still under the care of adults, the experience of an adolescent with the health care system and providers is less dependent on adults than that of younger children. Results from the Young Adult Health Care Survey (YAHCS) indicate that adolescents (including the publicly insured) are both valid and reliable reporters of certain aspects of preventive services.20,21 A survey of a sample of Medicaid-insured children aged 13 to 17 years using a CAHPS instrument showed a response rate of about 30%, with no difference according to mode.22 Teenagers enrolled in Medicaid and their parents were as willing to complete questionnaires about the adolescents’ health care as families with private health insurance. However, in both instances, population mobility and the quality of the contact information provided by the sponsoring agencies greatly hampered efforts to reach respondents, particularly in the Medicaid population.23 MEASUREMENT CONSIDERATIONS FOR CHILDREN Several aspects of children and the healthcare they require should be considered in assessing the quality of care they receive. 1. Age. Forrest and colleagues suggested the “4D model” for the unique attributes that should be considered in child
ACADEMIC PEDIATRICS
health services research or quality improvement,24 as well as when devising quality measurement strategies, including for patient experience. Three of these apply to quality measurement as related to family experiences with care. Children develop at a rapid rate, and their health and interactions with the health care system change over time.25 Measurement of patient and family experience of care across the age spectrum is clearly important. Children depend on parents, who act as the intermediaries in nearly all aspects of health care. Parent experience with the health care system is therefore important to assess, as it can be part of assessing the effectiveness of medical care delivered to children. Children’s demographic profiles have direct effects on their health and their health care. More than 1 in 5 children live in poverty.26,27 2. Care settings. Children receive health care in both the ambulatory and inpatient settings. Although a majority of children receive most of their care in the ambulatory setting, inpatient care includes settings such as surgical, neonatal, and intensive care, which are distinct and important. Hospitalized children represent a high-risk population for morbidity and mortality and account for a significant portion of pediatric health care costs.28 Patient experience of care measures are available for children for both settings and include aspects of care that are different from that in the other.11,29 Variation in parent perceptions of care are reported in both settings, providing valuable information for quality improvement.30,31 Soliciting parental perceptions of the process of inpatient care provides information for systems improvement.32 Ideally, assessment of patient and family experience with care would include assessment of both ambulatory and inpatient settings. 3. Healthy children versus children with chronic conditions. The Institute of Medicine1,33 suggests the following framework for measurement regarding the consumer perspective on health care needs: staying healthy: measures that assess helping people avoid illness and stay healthy through preventive care, reduction of health risks, early detection of illness, and education; getting better: measures that assess how the health care system helps people recover when they are sick or injured through appropriate and error-free treatment and follow-up; living with illness: measures focused on helping people with chronic conditions take care of themselves, control symptoms, avoid complications, and maintain daily activities through appropriate, error-free treatment and effective education and self-care support; changing needs: measures regarding caring for people and their families when needs change dramatically because of disability or terminal illness, with comprehensive and compassionate services, caregiver support, and hospice care. Accordingly, patient and family experience measures should encompass all these areas, including the care of healthy children, those that are acutely ill, and children with chronic conditions.
ACADEMIC PEDIATRICS
MEASURING FAMILY EXPERIENCES OF CARE
The aims of this report are as follows: to describe instruments currently in use for measuring patient/family experience of pediatric health care for quality reporting and/or accountability; to discuss recommendations made to the Agency for Healthcare Research and Quality (AHRQ) National Advisory Council for Healthcare Research and Quality Subcommittee on Children’s Healthcare Quality Measures for Medicaid and Child Health Insurance Programs (SNAC) based on this focused review of family experience measures for potential inclusion in the core measurement set; to identify gaps and barriers in measuring patient/family experience of pediatric health care; to provide recommendations for measure development.
METHODS We first conducted a review of quality measure databases, including the AHRQ National Quality Measures Clearinghouse, the National Quality Forum (NQF) endorsed measures, and the National Committee for Quality Assurance Healthcare Effectiveness Data and Information Set (HEDIS) to identify candidate instruments, as well as research their properties. Because the purpose of this report was to recommend quality measures for patient and family experience, we only reviewed measures that had been designed for quality measurement and accountability in the United States. We excluded measures used only for research. We assessed each measure for information regarding assessment of validity during measurement development. For reliability, we assessed domain internal consistency, as well as plan/ physician-level reliability when appropriate. For feasibility, we assessed on what scale (national, state, local) the instrument had been used, cost of administration (if available), and infrastructure needs (when applicable). Using Ovid search engine, we then conducted a literature search to further assess the identified instruments, as well as identify others. Search terms included patient satisfaction, patient experience, family experience, survey, questionnaires, and pediatric, and we limited our search to articles in the English language. One hundred seventy-one abstracts were identified and reviewed. We used instrument properties, including validity, reliability, and feasibility, as a basis for recommendations both for inclusion in the core measurement set and looked for gaps in measurement in regard to the spectrum of age, care settings, and health status (healthy vs chronic conditions) as a means to identify potential areas for measure development. RESULTS We identified patient/family experience instruments from the following 3 measure developers, which included assessment across the spectrum of ages, the ambulatory and inpatient settings, as well as of both healthy children and children with chronic illness (Table): 1) CAHPS, 2) Child and Adolescent Health Measurement Initiative
S61
(CAHMI) instruments, and 3) National Research Corporation Picker Pediatric Inpatient Survey (NRC Picker). CAHPS The CAHPS program is a public-private initiative to develop standardized surveys of patient and family experiences with health care. The CAHPS Health Plan Survey has been fielded since 1998 and includes versions for both public and commercially insured. The current CAHPS Consortium (CAHPS III) includes the Yale School of Public Health (previously referred to as the Harvard team) and RAND Corporation. AHRQ also contracts with Westat to support the CAHPS consortium, assist CAHPS users, and manage the National CAHPS Benchmarking Database. CAHPS surveys ask patients and consumers to report on their experiences with specific aspects of care, as well as provide a general rating of care for whatever level of the health care system (health plans, facilities, and providers) they are assessing. Patient/family reports about care are designed to be more specific, actionable, and objective compared to general ratings. Although CAHPS surveys initially focused on assessing care at the health plan level, additional surveys for measuring care at the level of primary care group practices and hospitals, as well as in behavioral health services and dental care, are now available.34,35 Because CAHPS surveys provide unique information about care delivery, their correlation with other clinical performance indicators and outcomes has been inconsistent.36–39 The CAHPS instrument development process includes major stakeholders such as consumers, clinicians, health care administrators, and accrediting bodies.40 CAHPS developers design items to assess aspects of care that they have identified as ones that consumers and patients value and are the best and/or only source of information, ie, the information cannot be gathered more effectively from other sources (chart review, etc). CAHPS instruments undergo cognitive and field testing and are designed to be applicable across heterogeneous populations, including the Medicaid and commercial insurance populations, across the age spectrum, and for vulnerable populations such as children with chronic conditions. CAHPS developers use the 0.70 internal consistency and reliability level as the minimum necessary for instrument use for “high stakes” purposes such as public reporting or payment. CAHPS instruments have demonstrated mode of administration (mail or telephone) does not appear to substantially influence response patterns in samples of the publicly and privately insured.41 The current CAHPS instruments for children are the Child Health Plan Survey and the Child Clinician & Group Survey. CAHPS CHILD MEDICAID HEALTH PLAN SURVEY The CAHPS Health Plan Survey 4.0, Child Medicaid version, is the most recent version in use for assessing patient/family experience in Medicaid programs. The core survey is 41 items, covers 4 domains (Table), and is available in English and Spanish. Parents of children (aged 17 years and younger) complete the survey. The
CO ET AL
S62
ACADEMIC PEDIATRICS
Table. Instruments for Measuring Patient/Family Experience of Care Instrument
Dimensions of Care Assessed
Areas Measured
CAHPS 4.0: Child Medicaid*
Access, utilization, communication, plan administration
Across ages, ambulatory setting, healthy children
CAHPS: Chronic Condition Supplement
Access to prescription medications, access to specialized services, familycentered care, personal doctor, shared decision making, getting needed information, coordination of care Access, communication, office staff, developmental surveillance, preventive care
Across ages, ambulatory setting, children with chronic conditions
CAHPS: Child Clinician & Group
Across ages, ambulatory setting, healthy children
Other Comments Supplemental items are available for areas such as behavioral health, communication, dental care, interpreters, and quality improvement
Supplemental items are available for areas including chronic conditions, doctor thoroughness, doctor knowledge of care from other providers, shared decision making
CAHMI†: Promoting Healthy Development Survey
Anticipatory guidance and parental Young children education; assessment of parental (aged 3–48 months), concerns; provision of specific ambulatory information; follow-up for children at setting, healthy children risk; administration of a standardized, parent-completed developmental and behavioral screening tool; assessment of psychosocial issues; assessment of smoking, substance abuse and safety; coordination of care; family-centered care; helpfulness of care; parental confidence; health information, resources in the community; provision of comprehensive care CAHMI: Young Adult Preventive screening and counseling on Adolescents (aged $14 years), Health Care Survey risky behaviors; preventive screening ambulatory setting, and counseling on sexual activity and healthy children STDs‡; preventive screening and counseling on weight, healthy diet, and exercise; preventive screening and counseling on emotional health and relationship issues; private and confidential care; helpfulness of counseling; communication and experience of care; health information National Research Partnership; confidence and trust; Across ages, inpatient Supplemental items are available Corporation Picker information, education, and setting for surgical and intensive care Pediatric Inpatient communication to parents; information, unit care, as well as overall satisfaction Survey education, and communication to child; coordination of care; physical comfort; continuity and transition *CAHPS ¼ Consumer Assessment of Health Providers Survey. †CAHMI ¼ Child and Adolescent Health Measurement Initiative. ‡STD ¼ sexually transmitted disease.
commercial and Medicaid versions of the survey are the same except for the time referent: the commercial questionnaire asks patients about their experiences in the previous 12 months, whereas the Medicaid instrument asks them about their experiences in the previous 6 months. In addition, a “children with chronic conditions” item set supplement is available. Both the Child Medicaid and the Chronic Condition Supplement have been endorsed by the NQF. In addition, supplemental items are available in several areas. The chronic conditions set for children was developed through a collaboration between the CAHPS Consortium, AHRQ, and the CAHMI Living with Illness Task Force.42
The set consists of a 5-item screener that classifies children with chronic conditions during the analysis stage after the survey has been administered, as well as a set of 24 supplemental items with special relevance to children with chronic conditions (Table). Items from the CAHPS–Children with Chronic Conditions are also included in the National Survey of Children with Special Health Care Needs. Survey properties. 1) Validity and reliability: Survey development included demonstration of strong construct validity,12 with correlations of the CAHPS global rating with willingness to recommend and re-enroll with the current health plan again. The most recent version of the CAHPS plan level survey showed internal consistency
ACADEMIC PEDIATRICS
reliability of 0.73 to 0.89 for the 4 domains (CAHPS Consortium, written communication, May 2010). 2) Feasibility: to obtain valid results for the child questionnaire that support comparisons of children with and without chronic conditions, the CAHPS Consortium recommends enriched sampling, with at least 300 responses from each category of children.43 Target response rates are 60% for commercial health plans and 50% for Medicaid, but lower rates are accepted. Experience indicates that the survey costs between $15.00 and $24.00 per completed survey. With 300 completes per health plan, the costs range from $4500 to $7200 for a mail-plus-telephone follow-up protocol. CAHPS CHILD CLINICIAN & GROUP SURVEY The CAHPS ambulatory care surveys44,45 (version 1.0) were officially endorsed by the NQF in 2007. Individual practice sites and groups have been shown to account for much of the explainable variation in patient reports across dimensions.46 Given that the unit of analysis is the clinician/group, one version of the survey is used for both Medicaid and the commercially insured. Recently, the Child Primary Care Questionnaire 2.0 (beta) was made available for use and is in the process of being submitted for NQF approval. The 54-item survey is available in English and Spanish. Parents of children aged 17 years and younger complete the survey. In addition to the traditional CAHPS reporting composites, the Child Clinician & Group Survey includes composites related to developmental surveillance and preventive care. The supplemental item set includes a new question about doctor communication and items to identify children with chronic conditions. Of note, Internet-based surveys were included in the testing of this instrument, but responses were insufficient for comparison with mail and phone responses. Survey properties. 1) Validity and reliability: Survey development included demonstration of strong construct validity, with composite scores strongly associated with overall ratings of the group and intention to switch.45 The developmental surveillance and preventive care composites had internal consistency and reliability greater than 0.70. The physician-level reliability of each composite was above 0.80. The estimated case-mix adjusted, physicianlevel reliabilities for samples of 25, 50, and 100 were 0.63, 0.77, and 0.87, respectively, for the development composite and 0.76, 0.86, and 0.93, respectively, for the preventive composite. 2) Feasibility: Measuring patient and family experience at the levels of clinician and group requires the mapping of individual patients with the providers and practices that provide care for them. The existence or need for compilation of this information determines the initial feasibility of measurement at the clinician group level. Survey administration cost and time. For applications of the survey intended to report or assess performance for individual physicians, the CAHPS Consortium recommends at least 45 completed surveys per physician.43 For applications of the survey intended to report or assess performance for a larger entity, such as a multisite medical group, with no
MEASURING FAMILY EXPERIENCES OF CARE
S63
interest in assessing individual physicians, the CAHPS Consortium recommends a minimum of 300 completed surveys per medical group. Costs associated with administering the CAHPS Clinician & Group Survey vary depending on the mode or mix of modes. Based on data from 3 test sites, CAHPS estimates costs per completed survey of $8.00 for mail administration and $11.00 for telephone administration. Cost per completed survey for mixed mode will be higher. With a target of 45 completes per physician, the cost ranges from $360 per physician to $495 per physician. Based on experience with other CAHPS surveys, the cost is likely to fall over time as larger scale surveying is done and vendors become more accustomed to the surveys. CAHMI INSTRUMENTS The CAHMI is a national initiative based out of Oregon Health and Science University, Department of Pediatrics, Portland, Oregon. Its mission is “To ensure that children, youth, and families are at the center of quality measurement and improvement efforts in order to advance a high quality consumer-centered health care system.”47 CAHMI develops, tests, and implements measurement strategies at the national, state, and practice levels. CAHMI disseminates its work to inform improvements in policy and practice. CAHMI designs instruments through their standardized multistage process, including focus groups with parents and/or children, review of existing relevant measures though literature review and key informant interviews, input from advisory groups (including clinicians, consumer representatives, public health experts, and researchers), as well as extensive cognitive and field testing. The current CAHMI instruments are the Promoting Healthy Development Survey (PHDS) and the YAHCS. PHDS PHDS is a 36-item parent survey that assesses whether young children (aged 3–48 months) are receiving nationally recommended preventive and developmental services (Table).48 The questionnaire asks about their experiences in the previous 12 months. It is endorsed by NQF for system-, plan-, and provider-level assessment of patient experience. Survey properties. 1) Validity and reliability: Psychometric analyses demonstrated that the PHDS scales showed good construct validity with mean factor loading of 0.69 and strong concurrent validity, with strong associations between parent reported receipt of anticipatory guidance and parenting self-efficacy, behaviors, and concern. Mean Cronbach a for survey domains was .80, with a range of .63 to .88. 2) Feasibility: It is available in English and Spanish, and it can be administered by mail, telephone, the Internet, and in pediatric offices (reduced item version with instruments of varied length). The PHDS has been used in both Medicaid and commercial health plans, and frontline health care providers. Items from the PHDS have also been implemented nationally through the National Survey of Early
S64
CO ET AL
Childhood Health and National Survey of Children’s Health, which is administered by the National Center for Health Statistics and is in the public domain. YAHCS The YAHCS is a 54-item teen survey that assesses whether young adults (aged 14 years and older) are receiving nationally recommended preventive services.20 The questionnaire asks about their experiences in the previous 12 months. The YAHCS items can be used to construct 8 composite measures and assesses quality of care (Table). It is endorsed by NQF. Survey properties. 1) Validity and reliability: The YAHCS scales showed good construct validity with mean factor loading of 0.64 and concurrent validity, with expected relationships between items and their related scales. Mean Cronbach a for survey domains was .77, with a range of .68 to .87. 2) Feasibility: The YAHCS can be administered by mail or telephone or as an online survey. It has been used in both Medicaid and commercial health plans and is a valid and reliable measure.2 NRC PICKER NRC Picker seeks to measure and improve the patient experience of care. They have developed measures both for children and adults and provide services for survey administration, data collection, and reporting. Their instruments are proprietary. The NRC Picker (NRC Picker, written communication, January 2010) was developed collaboratively in the late 1990s by the Picker Institute and Boston Children’s Hospital. They used focus groups comprising medical staff, nursing staff, and parents to modify the Picker Institute’s existing adult inpatient survey for assessing pediatric inpatient care. They field-tested candidate survey items with 300 parents of patients before finalizing the instrument. The instrument assesses 6 dimensions of care (Table). Items are also available to assess patient/family experience relative to intensive care or surgical services. The survey shows good criterion-related validity, with high correlation between the dimensions of care assessed and overall satisfaction, as well as to whether or not the parent recommended the hospital to their family or friends. Domain internal consistency and reliability range from 0.54 to 0.82, with 4 of the 6 dimensions >0.70. The Pediatric Inpatient Survey is used by many pediatric institutions, including many that are members of the National Association of Children’s Hospitals and Related Institutions. GAPS AND BARRIERS IN MEASUREMENT FOR CHILDREN BEHAVIORAL AND MENTAL HEALTH Behavioral and mental health services are becoming an increasingly important part of child health services, with gaps in care well documented in the literature.49 Although no validated widely disseminated patient and family experience measure for behavioral and mental health exists for children, a CAHPS instrument does exist for the adult population. The Experience of Care and Health Outcomes
ACADEMIC PEDIATRICS
(ECHO) Survey, the CAHPS survey of behavioral health services, measures the experiences of consumers with various aspects of mental health and substance abuse treatment and counseling services.50–53 The instrument was developed using the CAHPS instrument development process in collaboration with several groups, including the National Committee for Quality Assurance, several other national organizations, and behavioral health consumers. The survey allows sponsors to analyze the population by specific characteristics, including education, sex, and race. The survey was endorsed by NQF in July 2007. The current version of the adult survey is ECHO Survey 3.0. The ECHO Survey covers 2 types of organizations that are responsible for delivering behavioral health services: managed care organizations and managed behavioral health care organizations. The managed care organizations’ questionnaires consist of 63 core items that ask consumers about treatment, counseling, and administrative services, such as experiences filling out paperwork or finding information in written materials. Reporting measures include 5 composites (getting treatment quickly, communication, treatment and information from the plan, perceived improvement, information about treatment options), 2 global ratings, and 10 single-item measures. Both the managed care organizations’ and the managed behavioral health care organizations’ versions of the ECHO questionnaires may be administered to the parents or guardians of children who have received behavioral health services. Children are defined as people aged 17 years and younger at the time they received the services. Initial development of the pediatric instrument occurred as the adult instruments were developed, with mapping of adult-to-child items available on the CAHPS Web site. However, cognitive and pilot testing of the instruments is still necessary to complete instrument development, including the development of a set of reporting measures. CHALLENGES IN SURVEYING THE MEDICAID POPULATION Prior CAHPS work with the Medicaid population suggests the most effective data collection method to use is a mixed-mode approach.54 Regardless of mode, Medicaid populations require extra effort to attain a sufficient response rate55 and extra effort to track and locate recipients. Other considerations include setting a higher level of attempts for telephone surveys and adding an additional wave of mailing. The use of enhanced tracking tools such as operator assistance, reverse directories, and CD-ROM sources can increase the likelihood of locating Medicaid recipients, but they add length to the overall field period.
DISCUSSION RECOMMENDATIONS MADE TO SNAC Measures that are valid, reliable, and feasible exist for assessing patient and family experience of care across the spectrum of ages, care settings, and health status within the pediatric population. Compared to the CAHMI
ACADEMIC PEDIATRICS
measures, CAHPS measures have been used more extensively for quality measurement, quality improvement, and accountability. Two CAHMI instruments are focused on children during early development and on adolescent and young adult health care, providing more detail about services during those developmental periods. We recommended CAHPS for use more broadly because of its wider applicability across care settings, as well as the fact that it has already been used for several years, including for benchmarking. CAHPS instruments exist for assessing care in a valid and reliable way at the health plan and clinician group levels. These instruments assess care in the ambulatory setting, with parents acting as respondents in both cases. Items related to assessing care for children with chronic conditions are available for both of these instruments. Accordingly, we recommended that the CAHPS health plan and clinician group instruments for children be included in the core measurement set. Given that significantly more variation in patient and family experience is explained at the level of clinicians and groups compared to the plan level,56 we recommended that states that have the information and infrastructure necessary to assess care at the clinician group level do so instead of at the level of health plans. The ability to measure quality at the level of clinicians and groups can be useful for quality assessment and improvement broadly, including patient and family experience, care integration, and pay for performance. RECOMMENDATIONS FOR MEASURE DEVELOPMENT Gaps in measurement include assessment of behavioral and mental health and direct reports of care from the adolescent population. In the case of behavioral health and inpatient care, relevant CAHPS instruments exist for use in the adult population. We recommended that these be used as a basis for creating pediatric instruments for assessing care in these areas. For reports of care from the adolescent population, we encouraged states to consider use of the YAHCS to supplement their assessments from CAHPS instruments. States wanting more specific measures of clinical and patient-centered care and preventive and developmental services are encouraged to use the PHDS to supplement information garnered from CAHPS instruments. Another consideration for measure development is inpatient care. Although the NRC Picker instrument has good validity and reliability and has been used extensively by many institutions since its availability, one of its limitations is that its development and ownership is in the private sector. Thus, the ability to refine the survey is at the discretion of the proprietary owner. Adding supplemental items to assess specific areas would be difficult, and studies using data from a proprietary instrument such as the NRC Picker have not been extensive. Mechanisms need be developed to make proprietary instruments such as the NRC Picker more accessible for research, or developing a publically accessible instrument such as a pediatric version of the adult CAHPS Hospital Survey should be considered. The adults’ Hospital CAHPS has been endorsed by NQF, showing high construct validity and internal consistency
MEASURING FAMILY EXPERIENCES OF CARE
S65
and reliability.57 Modification of this instrument for use in children should be highly feasible given the CAHPS team’s prior experiences modifying adult instruments for use in the pediatric population. Development of a pediatric version of Hospital CAHPS also would facilitate an institution’s ability to benchmark pediatric services within a hospital with those that provide adult care. We also recommended further work be done to find ways to increase response rate and lower costs for receiving consumer feedback. Current response rates in the Medicaid population are low, raising concerns of sample bias. Both the CAHPS team and CAHMI have used the Internet for survey completion, and this and other modalities that could improve response merit further investigation. Finally, measurement developers should consider how existing patient/family experience measures developed for other purposes (ie, Patients Perception of Primary Care58 and Barriers to Care59) such as research can and should be used for assessing and improving quality of care. PATIENT AND FAMILY EXPERIENCE MEASURES RECOMMENDED BY SNAC The SNAC recommended core measurement set that went on for consideration by the Secretary of Health and Human Services included all of the measures recommended in this report. The final set, published on January 1, 2010, for public comment, included all these measures with the exception of the CAHPS Child Clinician & Group Survey. This measure was not selected because field experience with this survey was limited, with concerns about feasibility of implementation. Further experience with both the child and adult instruments will help in assessing the feasibility of more widespread use.60
ACKNOWLEDGMENTS Funding for this report was provided through purchase order HHSP233200900801P by the US Department of Health and Human Services Agency for Healthcare Research and Quality, with funds provided by its Centers for Medicare & Medicaid Services.
REFERENCES 1. Institute of Medicine. Envisioning the National Health Quality Report. Washington, DC: National Academies Press; 2001. 2. Institute of Medicine. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academies Press; 2001. 3. Buehler DM, Als H, Duffy FH, et al. Effectiveness of individualized developmental care for low-risk preterm infants: behavioral and electrophysiologic evidence. Pediatrics. 1995;96:924–932. 4. Clark NM, Gong M, Schork MA, et al. Impact of education for physicians on patient outcomes. Pediatrics. 1998;101:831–836. 5. Liptak GS. Enhancing patient compliance in pediatrics. Pediatrics. 1996;17:128–134. 6. Kaplan SH, Greenfield S, Ware JE Jr. Assessing the effects of physician-patient interactions on the outcomes of chronic disease. Med Care. 1989;27(suppl 3):S110–S127. 7. Institute of Medicine. Unequal Treatment: Confronting Racial and Ethnic Disparities in Health Care. Washington, DC: National Academies Press; 2001.
S66
CO ET AL
8. Zolnierek KB, Dimatteo MR. Physician communication and patient adherence to treatment: a meta-analysis. Med Care. 2009;47:826–834. 9. Harris-Kojetin LD, Fowler FJ Jr, Brown JA, et al. The use of cognitive testing to develop and evaluate CAHPS 1.0 core survey items. Consumer Assessment of Health Plans Study. Med Care. 1999;37(suppl 3):MS10–MS21. 10. Hays RD, Shaul JA, Williams VS, et al. Psychometric properties of the CAHPS 1.0 survey measures. Consumer Assessment of Health Plans Study. Med Care. 1999;37(suppl 3):MS22–MS31. 11. Hargraves JL, Hays RD, Cleary PD. Psychometric properties of the Consumer Assessment of Health Plans Study (CAHPS) 2.0 adult core survey. Health Serv Res. 2003;38(6 pt 1):1509–1527. 12. Shaul JA, Fowler FJ, Zaslavsky AM, et al. The impact of having parents report about both their own and their children’s experiences with health insurance plans. Med Care. 1999;37(suppl 3): MS59–MS68. 13. Rifkin L, Wolf MH, Lewis CC, Pantell RH. Children’s perceptions of physicians and medical care: two measures. J Pediatr Psychol. 1988; 13:247–254. 14. Schwab ME, Stone K. Conceptual and methodological issues in the evaluation of children’s satisfaction with their mental health care. Eval Program Plann. 1983;6:283–289. 15. Korsch BM, Gozzi EK, Francis V. Gaps in doctor-patient communication: doctor-patient interaction and patient satisfaction. Pediatrics. 1968;42:855–871. 16. Stewart TJ, Pantell RH, Dias JK, et al. Children as patients: a communications process study in family practice. J Fam Pract. 1981;13: 827–835. 17. Pantell RH, Stewart TJ, Dias JK, et al. Physician communication with children and parents. Pediatrics. 1982;70:396–402. 18. Homer CJ, Fowler FJ Jr, Gallagher PM, et al. The Consumer Assessment of Health Plan Study (CAHPS) survey of children’s health care. Jt Comm J Qual Improv. 1999;25:369–377. 19. Hedberg VA, Klein JD, Andresen E. Health counseling in adolescent preventive visits: effectiveness, current practices, and quality measurement. J Adolesc Health. 1998;23:344–353. 20. Bethell C, Klein J, Peck C. Assessing health system provision of adolescent preventive services: the Young Adult Health Care Survey. Med Care. 2001;39:478–490. 21. Santelli J, Klein J, Graff C, et al. Reliability of adolescent reporting of clinician counseling, health care use, and health behaviors. Med Care. 2002;40:26–37. 22. Gallagher PM, Fowler FJ, Elliott D. Collecting information about the health experiences of publicly insured adolescents. In: Cynamon ML, Kulka RA, eds. Seventh Conference on Health Survey Research Methods. Hyattsville, Md: National Center for Health Statistics; 1999:3–9. DHHS Publication No. (PHS) 01–1013. 23. Gallagher PM, Fowler FJ Jr. Collecting information about the health care experiences of adolescents. In: Proceedings of the Survey Research Methods Section, 1998. Alexandria, Va: American Statistical Association; 1998. 878–882. 24. Forrest CB, Simpson L, Clancy C. Child health services research. Challenges and opportunities. JAMA. 1997;277:1787–1793. 25. Schor EL. Use of health care services by children and diagnoses received during presumably stressful life transitions. Pediatrics. 1986;77:834–841. 26. General Accounting Office. Infants and Toddlers: Dramatic Increases in Numbers Living in Poverty. Washington, DC: General Accounting Office. GAO Pub No GAU/HEHS-94–74. Available at: http://archive. gao.gov/t2pbat3/151302.pdf; 1994. Accessed October 22, 2010. 27. US Department of Commerce, Bureau of the Census. Poverty in the United States, 1992. Current Population Reports. Washington, DC: US Department of Commerce, Bureau of the Census; 1993. 28. Chevarley FM, Owens PL, Zodet MW, et al. Health care for children and youth in the United States: annual report on patterns of coverage, utilization, quality, and expenditures by a county level of urban influence. Ambul Pediatr. 2006;6:241–264. 29. Homer CJ, Marino B, Cleary PD, et al. Quality of care at a children’s hospital. Arch Pediatr Adolesc Med. 1999;153:1123–1129.
ACADEMIC PEDIATRICS 30. Co JP, Ferris TG, Marino BL, et al. Are hospital characteristics associated with parental views of pediatric inpatient care quality? Pediatrics. 2003;111:308–314. 31. Co JP, Macdonald E, Yucel RM, Ferris TG. Practice variation in parental assessment of pediatric ambulatory care. Acad Pediatr. 2009;9:47–52. 32. Cleary PD. The increasing importance of patient surveys. Br Med J. 1999;319:720–721. 33. Bethell C. Patient-Centered Care Measures for the National Healthcare Quality Report. Portland, Ore: Foundation for Accountability (FACCT). Available at: http://www.policyarchive.org/handle/10207/ bitstreams/95530.pdf. Accessed October 22, 2010. 34. Solomon LS, Zaslavsky AM, Landon BE, Cleary PD. Variation in patient-reported quality among health care organizations. Health Care Financ Rev. 2002;23(4):85–100. 35. Nelson EC, Gentry MA, Mook KH, et al. How many patients are needed to provide reliable evaluations of individual clinicians? Med Care. 2004;42:259–266. 36. Fremont AM, Cleary PD, Hargraves JL, et al. Patient-centered processes of care and long-term outcomes of myocardial infarction. J Gen Intern Med. 2001;16:800–808. 37. Schneider EC, Zaslavsky AM, Landon BE, et al. National quality monitoring of Medicare health plans: the relationship between enrollees’ reports and the quality of clinical care. Med Care. 2001;39:1313–1325. 38. Chang JT, Hays RD, Shekelle PG, et al. Patients’ global ratings of their health care are not associated with the technical quality of their care. Ann Intern Med. 2006;144:665–672. 39. Rao M, Clarke A, Sanderson C, Hammersley R. Patients’ own assessments of quality of primary care compared with objective records based measures of technical quality of care: cross sectional study. Br Med J. 2006;333:19. 40. US Dept of Health and Human Services. CAHPS overview. Available at: https://www.cahps.ahrq.gov/content/cahpsOverview/OVER_ Intro.asp?p¼101&s¼1. Accessed October 22, 2010. 41. Fowler FJ Jr, Gallagher PM, Nederend S. Comparing telephone and mail responses to the CAHPS survey instrument. Med Care. 1999; 37:MS41–MS49. 42. The Child and Adolescent Health Measurement Initiative. Children with special health care needs: overview. Available at: http://cahmi. org/pages/Sections.aspx?section¼10. Accessed October 22, 2010. 43. US Dept of Health and Human Services. CAHPS Health Plan Survey and Reporting Kit 2008. Available at: https://www.cahps. ahrq.gov/CAHPSkit/Healthplan/HPChooseQx2.asp. Accessed October 22, 2010. 44. Gallagher P, Ding L, Ham HP, et al. Development of a new patientbased measure of pediatric ambulatory care. Pediatrics. 2009;124: 1348–1354. 45. Solomon LS, Hays RD, Zaslavsky AM, et al. Psychometric properties of a group-level Consumer Assessment of Health Plans Study (CAHPS) instrument. Med Care. 2005;43:53–60. 46. Solomon LS, Zaslavsky AM, Landon BE, Cleary PD. Variation in patient-reported quality among health care organizations. Health Care Financ Rev. 2002;23:85–100. 47. CAHPS Clinician and Group Survey and Reporting Kit 2008, Fielding the Survey. Available at: https://www.cahps.ahrq.gov/CAHPSkit/ Healthplan/HPChooseQx2.asp. Accessed October 22, 2010. 48. Bethell C, Peck C, Schor E. Assessing health system provision of well-child care: the Promoting Healthy Development Survey. Pediatrics. 2001;107:1084–1094. 49. Institute of Medicine. Preventing Mental, Emotional, and Behavioral Disorders Among Young People: Progress and Possibilities. Washington, DC: National Academies Press; 2009. 50. Eselius LL, Zaslavsky AM, Huskamp HA, et al. Case-mix adjustment of consumer reports about managed behavioral health care and health plans. Health Serv Res. 2008;43:2014–2032. 51. Eisen SV, Shaul JA, Clarridge B, et al. Development of a consumer survey for behavioral health services. Psychiatr Serv. 1999;50:793–798. 52. Shaul JA, Eisen SV, Stringfellow VL, et al. Use of consumer ratings for quality improvement in behavioral health insurance plans. Jt Comm J Qual Improv. 2001;27:216–229.
ACADEMIC PEDIATRICS 53. Eisen SV, Shaul JA, Leff HS, et al. Toward a national consumer survey: evaluation of the CABHS and MHSIP instruments. J Behav Health Serv Res. 2001;28:347–369. 54. Brown JA, Nederend SE, Hays RD, et al. Special issues in assessing care of Medicaid recipients. Med Care. 1999;37(suppl 3):MS79–MS88. 55. Dillman D, Sinclair M, Clark J. Effects of questionnaire length, respondent-friendly design, and a difficult question on response rates for occupant-addressed census mail surveys. Public Opin Q. 1993;57:289. 56. Safran DG, Karp M, Coltin K, et al. Measuring patients’ experiences with individual primary care physicians. Results of a statewide demonstration project. J Gen Intern Med. 2006;21:13–21.
MEASURING FAMILY EXPERIENCES OF CARE
S67
57. CAHPS Hospital Survey. Available at: http://hcahpsonline.org. Accessed October 22, 2010. 58. Seid M, Stevens GD, Varni JW. Parents’ perceptions of pediatric primary care quality: effects of race/ethnicity, language, and access. Health Serv Res. 2003;38:1009–1031. 59. Sobo EJ, Seid M, Reyes Gelhard L. Parent-identified barriers to pediatric health care: a process-oriented model. Health Serv Res. 2006;41:148–172. 60. 74 Federal Register 68846–68849 (2009). Available at: http://www. gpo.gov/fdsys/pkg/FR-2009–12–29/html/E9–30802.htm. Accessed November 9, 2010.
ABSTRACT BACKGROUND: The Institute of Medicine considers patient
Survey (NRC Picker). We identified the need for matching patients with providers and groups as a barrier for widespread use of the CAHPS pediatric clinician & group instrument. CONCLUSIONS: We recommended to the National Advisory Council for Healthcare Research and Quality Subcommittee on Children’s Healthcare Quality Measures for Medicaid and Child Health Insurance Programs (SNAC) the CAHPS Child Medicaid 4.0 and pediatric Clinician & Group Survey for inclusion in the initial recommended list of core measures for voluntary use by Medicaid and CHIP. The Clinician and Group Survey was not included in the list posted for public comment due to concerns at that time (December 2009) about feasibility. We also recommended that development of a child version of the CAHPS behavioral and mental health survey now used in the adult population and of a pediatric hospital CAHPS measure be considered high priorities for development in the next phase of Children’s Health Insurance Program Reauthorization Act measurement activity. This phase should also explore methods to increase response rates and lower costs of obtaining consumer feedback.
centeredness a core dimension of quality. Several patient/family surveys exist to assess pediatric health care. The Children’s Health Insurance Program Reauthorization Act mandates strengthening quality measurement for children, including for patient/family experience of care. OBJECTIVES: The aim of this study was to determine what instruments exist for measuring patient/family experience of pediatric health care and which should be included in the core measurement set for assessing Medicaid and the Children’s Health Insurance Program (CHIP) programs; to identify gaps in measurement; and to provide recommendations for measure development. METHODS: We developed a conceptual framework for measuring patient/family experience of care. We conducted a review of national measure clearinghouses and of the literature to assess validity, reliability, and feasibility of existing measures, and how these measures address the conceptual framework. RESULTS: We found valid and reliable instruments for measuring patient/family experience of care include the Consumer Assessment of Healthcare Providers and Systems (CAHPS) instruments, Promoting Healthy Development Survey (PHDS), Young Adult Health Care Survey (YAHCS), and the National Research Corporation Picker Pediatric Inpatient
KEYWORDS: patient centeredness; patient experience; quality of care; surveys ACADEMIC PEDIATRICS 2011;11:S59–S67
ON FEBRUARY 4, 2009, the Children’s Health Insurance Program Reauthorization Act (Public Law 111-3) was signed into law by President Obama, reauthorizing the Children’s Health Insurance Program (CHIP). Included in the law was a mandate to strengthen quality of care and health outcomes for children enrolled in Medicaid and CHIP programs, and specific language directing the Secretary of Health and Human Services to identify a recommended core set of child health quality measures for voluntary use by state Medicaid and CHIP programs by January 1, 2010. These measures, to include assessment of patient/family experience of care, should provide information on the quality of health care and coverage of children across the age and health spectrum and ACADEMIC PEDIATRICS Copyright ª 2011 by Academic Pediatric Association
provide helpful information for improving the quality of care for children. The Institute of Medicine considers patient centeredness a core dimension of quality, one that should be included in assessing the quality of health care delivered in the United States and that can only be assessed through patient reports of care.1 Patient-centered heath care includes establishing a partnership between providers, patients, and their families, as well as the patient experience itself. Improvements in health care design and delivery should include making the system more patient centered.2 Patient-centered approaches to care are associated with improvements in care, including for chronic care management.3–6 In particular, effective communication is associated with improved patient satisfaction and adherence.7,8
S59
Volume 11, Number 3S May–June 2011
S60
CO ET AL
CONCEPTUAL FRAMEWORK FOR MEASURING PATIENT/ FAMILY EXPERIENCE ASPECTS OF CARE THAT CAN BE REPORTED AND MEASURED The development of the initial versions of the Consumer Assessment of Healthcare Providers and Systems (CAHPS) instruments (including the child survey) for assessing health plans provided a critical foundation for measuring patient/ family experience.9–11 Focus groups and extensive cognitive testing revealed that domains such as access to care, timeliness of care, communication, office staff, and health plan customer service could validly and reliably be reported by adult respondents. Further work, including literature review and interviews with experts in pediatric medical care and health plan satisfaction, determined that the content of the child survey should differ from the content of the adult survey.12 Some additional areas were identified, including the child’s interactions and communication with health care professionals; preventive care; and care for the child’s growth, development, or behavior. REPORTING ON CHILDREN’S HEALTH CARE Survey data about children’s health care is usually collected from parents rather than directly from children, for both logistic and methodological reasons.13,14 Issues such as obtaining informed consent, contacting children, and interviewing them are substantial challenges for obtaining survey responses in the pediatric population.15–17 For example, children aged younger than 12 years tend to respond positively to survey questions regardless of their content.18 The limited and changing cognitive ability of children limits the complexity of item wording. These types of issues have prompted most to assess children’s health care by using parent report. Measurement of patient/family experience with health care is a particular challenge in the adolescent population.19 Although usually still under the care of adults, the experience of an adolescent with the health care system and providers is less dependent on adults than that of younger children. Results from the Young Adult Health Care Survey (YAHCS) indicate that adolescents (including the publicly insured) are both valid and reliable reporters of certain aspects of preventive services.20,21 A survey of a sample of Medicaid-insured children aged 13 to 17 years using a CAHPS instrument showed a response rate of about 30%, with no difference according to mode.22 Teenagers enrolled in Medicaid and their parents were as willing to complete questionnaires about the adolescents’ health care as families with private health insurance. However, in both instances, population mobility and the quality of the contact information provided by the sponsoring agencies greatly hampered efforts to reach respondents, particularly in the Medicaid population.23 MEASUREMENT CONSIDERATIONS FOR CHILDREN Several aspects of children and the healthcare they require should be considered in assessing the quality of care they receive. 1. Age. Forrest and colleagues suggested the “4D model” for the unique attributes that should be considered in child
ACADEMIC PEDIATRICS
health services research or quality improvement,24 as well as when devising quality measurement strategies, including for patient experience. Three of these apply to quality measurement as related to family experiences with care. Children develop at a rapid rate, and their health and interactions with the health care system change over time.25 Measurement of patient and family experience of care across the age spectrum is clearly important. Children depend on parents, who act as the intermediaries in nearly all aspects of health care. Parent experience with the health care system is therefore important to assess, as it can be part of assessing the effectiveness of medical care delivered to children. Children’s demographic profiles have direct effects on their health and their health care. More than 1 in 5 children live in poverty.26,27 2. Care settings. Children receive health care in both the ambulatory and inpatient settings. Although a majority of children receive most of their care in the ambulatory setting, inpatient care includes settings such as surgical, neonatal, and intensive care, which are distinct and important. Hospitalized children represent a high-risk population for morbidity and mortality and account for a significant portion of pediatric health care costs.28 Patient experience of care measures are available for children for both settings and include aspects of care that are different from that in the other.11,29 Variation in parent perceptions of care are reported in both settings, providing valuable information for quality improvement.30,31 Soliciting parental perceptions of the process of inpatient care provides information for systems improvement.32 Ideally, assessment of patient and family experience with care would include assessment of both ambulatory and inpatient settings. 3. Healthy children versus children with chronic conditions. The Institute of Medicine1,33 suggests the following framework for measurement regarding the consumer perspective on health care needs: staying healthy: measures that assess helping people avoid illness and stay healthy through preventive care, reduction of health risks, early detection of illness, and education; getting better: measures that assess how the health care system helps people recover when they are sick or injured through appropriate and error-free treatment and follow-up; living with illness: measures focused on helping people with chronic conditions take care of themselves, control symptoms, avoid complications, and maintain daily activities through appropriate, error-free treatment and effective education and self-care support; changing needs: measures regarding caring for people and their families when needs change dramatically because of disability or terminal illness, with comprehensive and compassionate services, caregiver support, and hospice care. Accordingly, patient and family experience measures should encompass all these areas, including the care of healthy children, those that are acutely ill, and children with chronic conditions.
ACADEMIC PEDIATRICS
MEASURING FAMILY EXPERIENCES OF CARE
The aims of this report are as follows: to describe instruments currently in use for measuring patient/family experience of pediatric health care for quality reporting and/or accountability; to discuss recommendations made to the Agency for Healthcare Research and Quality (AHRQ) National Advisory Council for Healthcare Research and Quality Subcommittee on Children’s Healthcare Quality Measures for Medicaid and Child Health Insurance Programs (SNAC) based on this focused review of family experience measures for potential inclusion in the core measurement set; to identify gaps and barriers in measuring patient/family experience of pediatric health care; to provide recommendations for measure development.
METHODS We first conducted a review of quality measure databases, including the AHRQ National Quality Measures Clearinghouse, the National Quality Forum (NQF) endorsed measures, and the National Committee for Quality Assurance Healthcare Effectiveness Data and Information Set (HEDIS) to identify candidate instruments, as well as research their properties. Because the purpose of this report was to recommend quality measures for patient and family experience, we only reviewed measures that had been designed for quality measurement and accountability in the United States. We excluded measures used only for research. We assessed each measure for information regarding assessment of validity during measurement development. For reliability, we assessed domain internal consistency, as well as plan/ physician-level reliability when appropriate. For feasibility, we assessed on what scale (national, state, local) the instrument had been used, cost of administration (if available), and infrastructure needs (when applicable). Using Ovid search engine, we then conducted a literature search to further assess the identified instruments, as well as identify others. Search terms included patient satisfaction, patient experience, family experience, survey, questionnaires, and pediatric, and we limited our search to articles in the English language. One hundred seventy-one abstracts were identified and reviewed. We used instrument properties, including validity, reliability, and feasibility, as a basis for recommendations both for inclusion in the core measurement set and looked for gaps in measurement in regard to the spectrum of age, care settings, and health status (healthy vs chronic conditions) as a means to identify potential areas for measure development. RESULTS We identified patient/family experience instruments from the following 3 measure developers, which included assessment across the spectrum of ages, the ambulatory and inpatient settings, as well as of both healthy children and children with chronic illness (Table): 1) CAHPS, 2) Child and Adolescent Health Measurement Initiative
S61
(CAHMI) instruments, and 3) National Research Corporation Picker Pediatric Inpatient Survey (NRC Picker). CAHPS The CAHPS program is a public-private initiative to develop standardized surveys of patient and family experiences with health care. The CAHPS Health Plan Survey has been fielded since 1998 and includes versions for both public and commercially insured. The current CAHPS Consortium (CAHPS III) includes the Yale School of Public Health (previously referred to as the Harvard team) and RAND Corporation. AHRQ also contracts with Westat to support the CAHPS consortium, assist CAHPS users, and manage the National CAHPS Benchmarking Database. CAHPS surveys ask patients and consumers to report on their experiences with specific aspects of care, as well as provide a general rating of care for whatever level of the health care system (health plans, facilities, and providers) they are assessing. Patient/family reports about care are designed to be more specific, actionable, and objective compared to general ratings. Although CAHPS surveys initially focused on assessing care at the health plan level, additional surveys for measuring care at the level of primary care group practices and hospitals, as well as in behavioral health services and dental care, are now available.34,35 Because CAHPS surveys provide unique information about care delivery, their correlation with other clinical performance indicators and outcomes has been inconsistent.36–39 The CAHPS instrument development process includes major stakeholders such as consumers, clinicians, health care administrators, and accrediting bodies.40 CAHPS developers design items to assess aspects of care that they have identified as ones that consumers and patients value and are the best and/or only source of information, ie, the information cannot be gathered more effectively from other sources (chart review, etc). CAHPS instruments undergo cognitive and field testing and are designed to be applicable across heterogeneous populations, including the Medicaid and commercial insurance populations, across the age spectrum, and for vulnerable populations such as children with chronic conditions. CAHPS developers use the 0.70 internal consistency and reliability level as the minimum necessary for instrument use for “high stakes” purposes such as public reporting or payment. CAHPS instruments have demonstrated mode of administration (mail or telephone) does not appear to substantially influence response patterns in samples of the publicly and privately insured.41 The current CAHPS instruments for children are the Child Health Plan Survey and the Child Clinician & Group Survey. CAHPS CHILD MEDICAID HEALTH PLAN SURVEY The CAHPS Health Plan Survey 4.0, Child Medicaid version, is the most recent version in use for assessing patient/family experience in Medicaid programs. The core survey is 41 items, covers 4 domains (Table), and is available in English and Spanish. Parents of children (aged 17 years and younger) complete the survey. The
CO ET AL
S62
ACADEMIC PEDIATRICS
Table. Instruments for Measuring Patient/Family Experience of Care Instrument
Dimensions of Care Assessed
Areas Measured
CAHPS 4.0: Child Medicaid*
Access, utilization, communication, plan administration
Across ages, ambulatory setting, healthy children
CAHPS: Chronic Condition Supplement
Access to prescription medications, access to specialized services, familycentered care, personal doctor, shared decision making, getting needed information, coordination of care Access, communication, office staff, developmental surveillance, preventive care
Across ages, ambulatory setting, children with chronic conditions
CAHPS: Child Clinician & Group
Across ages, ambulatory setting, healthy children
Other Comments Supplemental items are available for areas such as behavioral health, communication, dental care, interpreters, and quality improvement
Supplemental items are available for areas including chronic conditions, doctor thoroughness, doctor knowledge of care from other providers, shared decision making
CAHMI†: Promoting Healthy Development Survey
Anticipatory guidance and parental Young children education; assessment of parental (aged 3–48 months), concerns; provision of specific ambulatory information; follow-up for children at setting, healthy children risk; administration of a standardized, parent-completed developmental and behavioral screening tool; assessment of psychosocial issues; assessment of smoking, substance abuse and safety; coordination of care; family-centered care; helpfulness of care; parental confidence; health information, resources in the community; provision of comprehensive care CAHMI: Young Adult Preventive screening and counseling on Adolescents (aged $14 years), Health Care Survey risky behaviors; preventive screening ambulatory setting, and counseling on sexual activity and healthy children STDs‡; preventive screening and counseling on weight, healthy diet, and exercise; preventive screening and counseling on emotional health and relationship issues; private and confidential care; helpfulness of counseling; communication and experience of care; health information National Research Partnership; confidence and trust; Across ages, inpatient Supplemental items are available Corporation Picker information, education, and setting for surgical and intensive care Pediatric Inpatient communication to parents; information, unit care, as well as overall satisfaction Survey education, and communication to child; coordination of care; physical comfort; continuity and transition *CAHPS ¼ Consumer Assessment of Health Providers Survey. †CAHMI ¼ Child and Adolescent Health Measurement Initiative. ‡STD ¼ sexually transmitted disease.
commercial and Medicaid versions of the survey are the same except for the time referent: the commercial questionnaire asks patients about their experiences in the previous 12 months, whereas the Medicaid instrument asks them about their experiences in the previous 6 months. In addition, a “children with chronic conditions” item set supplement is available. Both the Child Medicaid and the Chronic Condition Supplement have been endorsed by the NQF. In addition, supplemental items are available in several areas. The chronic conditions set for children was developed through a collaboration between the CAHPS Consortium, AHRQ, and the CAHMI Living with Illness Task Force.42
The set consists of a 5-item screener that classifies children with chronic conditions during the analysis stage after the survey has been administered, as well as a set of 24 supplemental items with special relevance to children with chronic conditions (Table). Items from the CAHPS–Children with Chronic Conditions are also included in the National Survey of Children with Special Health Care Needs. Survey properties. 1) Validity and reliability: Survey development included demonstration of strong construct validity,12 with correlations of the CAHPS global rating with willingness to recommend and re-enroll with the current health plan again. The most recent version of the CAHPS plan level survey showed internal consistency
ACADEMIC PEDIATRICS
reliability of 0.73 to 0.89 for the 4 domains (CAHPS Consortium, written communication, May 2010). 2) Feasibility: to obtain valid results for the child questionnaire that support comparisons of children with and without chronic conditions, the CAHPS Consortium recommends enriched sampling, with at least 300 responses from each category of children.43 Target response rates are 60% for commercial health plans and 50% for Medicaid, but lower rates are accepted. Experience indicates that the survey costs between $15.00 and $24.00 per completed survey. With 300 completes per health plan, the costs range from $4500 to $7200 for a mail-plus-telephone follow-up protocol. CAHPS CHILD CLINICIAN & GROUP SURVEY The CAHPS ambulatory care surveys44,45 (version 1.0) were officially endorsed by the NQF in 2007. Individual practice sites and groups have been shown to account for much of the explainable variation in patient reports across dimensions.46 Given that the unit of analysis is the clinician/group, one version of the survey is used for both Medicaid and the commercially insured. Recently, the Child Primary Care Questionnaire 2.0 (beta) was made available for use and is in the process of being submitted for NQF approval. The 54-item survey is available in English and Spanish. Parents of children aged 17 years and younger complete the survey. In addition to the traditional CAHPS reporting composites, the Child Clinician & Group Survey includes composites related to developmental surveillance and preventive care. The supplemental item set includes a new question about doctor communication and items to identify children with chronic conditions. Of note, Internet-based surveys were included in the testing of this instrument, but responses were insufficient for comparison with mail and phone responses. Survey properties. 1) Validity and reliability: Survey development included demonstration of strong construct validity, with composite scores strongly associated with overall ratings of the group and intention to switch.45 The developmental surveillance and preventive care composites had internal consistency and reliability greater than 0.70. The physician-level reliability of each composite was above 0.80. The estimated case-mix adjusted, physicianlevel reliabilities for samples of 25, 50, and 100 were 0.63, 0.77, and 0.87, respectively, for the development composite and 0.76, 0.86, and 0.93, respectively, for the preventive composite. 2) Feasibility: Measuring patient and family experience at the levels of clinician and group requires the mapping of individual patients with the providers and practices that provide care for them. The existence or need for compilation of this information determines the initial feasibility of measurement at the clinician group level. Survey administration cost and time. For applications of the survey intended to report or assess performance for individual physicians, the CAHPS Consortium recommends at least 45 completed surveys per physician.43 For applications of the survey intended to report or assess performance for a larger entity, such as a multisite medical group, with no
MEASURING FAMILY EXPERIENCES OF CARE
S63
interest in assessing individual physicians, the CAHPS Consortium recommends a minimum of 300 completed surveys per medical group. Costs associated with administering the CAHPS Clinician & Group Survey vary depending on the mode or mix of modes. Based on data from 3 test sites, CAHPS estimates costs per completed survey of $8.00 for mail administration and $11.00 for telephone administration. Cost per completed survey for mixed mode will be higher. With a target of 45 completes per physician, the cost ranges from $360 per physician to $495 per physician. Based on experience with other CAHPS surveys, the cost is likely to fall over time as larger scale surveying is done and vendors become more accustomed to the surveys. CAHMI INSTRUMENTS The CAHMI is a national initiative based out of Oregon Health and Science University, Department of Pediatrics, Portland, Oregon. Its mission is “To ensure that children, youth, and families are at the center of quality measurement and improvement efforts in order to advance a high quality consumer-centered health care system.”47 CAHMI develops, tests, and implements measurement strategies at the national, state, and practice levels. CAHMI disseminates its work to inform improvements in policy and practice. CAHMI designs instruments through their standardized multistage process, including focus groups with parents and/or children, review of existing relevant measures though literature review and key informant interviews, input from advisory groups (including clinicians, consumer representatives, public health experts, and researchers), as well as extensive cognitive and field testing. The current CAHMI instruments are the Promoting Healthy Development Survey (PHDS) and the YAHCS. PHDS PHDS is a 36-item parent survey that assesses whether young children (aged 3–48 months) are receiving nationally recommended preventive and developmental services (Table).48 The questionnaire asks about their experiences in the previous 12 months. It is endorsed by NQF for system-, plan-, and provider-level assessment of patient experience. Survey properties. 1) Validity and reliability: Psychometric analyses demonstrated that the PHDS scales showed good construct validity with mean factor loading of 0.69 and strong concurrent validity, with strong associations between parent reported receipt of anticipatory guidance and parenting self-efficacy, behaviors, and concern. Mean Cronbach a for survey domains was .80, with a range of .63 to .88. 2) Feasibility: It is available in English and Spanish, and it can be administered by mail, telephone, the Internet, and in pediatric offices (reduced item version with instruments of varied length). The PHDS has been used in both Medicaid and commercial health plans, and frontline health care providers. Items from the PHDS have also been implemented nationally through the National Survey of Early
S64
CO ET AL
Childhood Health and National Survey of Children’s Health, which is administered by the National Center for Health Statistics and is in the public domain. YAHCS The YAHCS is a 54-item teen survey that assesses whether young adults (aged 14 years and older) are receiving nationally recommended preventive services.20 The questionnaire asks about their experiences in the previous 12 months. The YAHCS items can be used to construct 8 composite measures and assesses quality of care (Table). It is endorsed by NQF. Survey properties. 1) Validity and reliability: The YAHCS scales showed good construct validity with mean factor loading of 0.64 and concurrent validity, with expected relationships between items and their related scales. Mean Cronbach a for survey domains was .77, with a range of .68 to .87. 2) Feasibility: The YAHCS can be administered by mail or telephone or as an online survey. It has been used in both Medicaid and commercial health plans and is a valid and reliable measure.2 NRC PICKER NRC Picker seeks to measure and improve the patient experience of care. They have developed measures both for children and adults and provide services for survey administration, data collection, and reporting. Their instruments are proprietary. The NRC Picker (NRC Picker, written communication, January 2010) was developed collaboratively in the late 1990s by the Picker Institute and Boston Children’s Hospital. They used focus groups comprising medical staff, nursing staff, and parents to modify the Picker Institute’s existing adult inpatient survey for assessing pediatric inpatient care. They field-tested candidate survey items with 300 parents of patients before finalizing the instrument. The instrument assesses 6 dimensions of care (Table). Items are also available to assess patient/family experience relative to intensive care or surgical services. The survey shows good criterion-related validity, with high correlation between the dimensions of care assessed and overall satisfaction, as well as to whether or not the parent recommended the hospital to their family or friends. Domain internal consistency and reliability range from 0.54 to 0.82, with 4 of the 6 dimensions >0.70. The Pediatric Inpatient Survey is used by many pediatric institutions, including many that are members of the National Association of Children’s Hospitals and Related Institutions. GAPS AND BARRIERS IN MEASUREMENT FOR CHILDREN BEHAVIORAL AND MENTAL HEALTH Behavioral and mental health services are becoming an increasingly important part of child health services, with gaps in care well documented in the literature.49 Although no validated widely disseminated patient and family experience measure for behavioral and mental health exists for children, a CAHPS instrument does exist for the adult population. The Experience of Care and Health Outcomes
ACADEMIC PEDIATRICS
(ECHO) Survey, the CAHPS survey of behavioral health services, measures the experiences of consumers with various aspects of mental health and substance abuse treatment and counseling services.50–53 The instrument was developed using the CAHPS instrument development process in collaboration with several groups, including the National Committee for Quality Assurance, several other national organizations, and behavioral health consumers. The survey allows sponsors to analyze the population by specific characteristics, including education, sex, and race. The survey was endorsed by NQF in July 2007. The current version of the adult survey is ECHO Survey 3.0. The ECHO Survey covers 2 types of organizations that are responsible for delivering behavioral health services: managed care organizations and managed behavioral health care organizations. The managed care organizations’ questionnaires consist of 63 core items that ask consumers about treatment, counseling, and administrative services, such as experiences filling out paperwork or finding information in written materials. Reporting measures include 5 composites (getting treatment quickly, communication, treatment and information from the plan, perceived improvement, information about treatment options), 2 global ratings, and 10 single-item measures. Both the managed care organizations’ and the managed behavioral health care organizations’ versions of the ECHO questionnaires may be administered to the parents or guardians of children who have received behavioral health services. Children are defined as people aged 17 years and younger at the time they received the services. Initial development of the pediatric instrument occurred as the adult instruments were developed, with mapping of adult-to-child items available on the CAHPS Web site. However, cognitive and pilot testing of the instruments is still necessary to complete instrument development, including the development of a set of reporting measures. CHALLENGES IN SURVEYING THE MEDICAID POPULATION Prior CAHPS work with the Medicaid population suggests the most effective data collection method to use is a mixed-mode approach.54 Regardless of mode, Medicaid populations require extra effort to attain a sufficient response rate55 and extra effort to track and locate recipients. Other considerations include setting a higher level of attempts for telephone surveys and adding an additional wave of mailing. The use of enhanced tracking tools such as operator assistance, reverse directories, and CD-ROM sources can increase the likelihood of locating Medicaid recipients, but they add length to the overall field period.
DISCUSSION RECOMMENDATIONS MADE TO SNAC Measures that are valid, reliable, and feasible exist for assessing patient and family experience of care across the spectrum of ages, care settings, and health status within the pediatric population. Compared to the CAHMI
ACADEMIC PEDIATRICS
measures, CAHPS measures have been used more extensively for quality measurement, quality improvement, and accountability. Two CAHMI instruments are focused on children during early development and on adolescent and young adult health care, providing more detail about services during those developmental periods. We recommended CAHPS for use more broadly because of its wider applicability across care settings, as well as the fact that it has already been used for several years, including for benchmarking. CAHPS instruments exist for assessing care in a valid and reliable way at the health plan and clinician group levels. These instruments assess care in the ambulatory setting, with parents acting as respondents in both cases. Items related to assessing care for children with chronic conditions are available for both of these instruments. Accordingly, we recommended that the CAHPS health plan and clinician group instruments for children be included in the core measurement set. Given that significantly more variation in patient and family experience is explained at the level of clinicians and groups compared to the plan level,56 we recommended that states that have the information and infrastructure necessary to assess care at the clinician group level do so instead of at the level of health plans. The ability to measure quality at the level of clinicians and groups can be useful for quality assessment and improvement broadly, including patient and family experience, care integration, and pay for performance. RECOMMENDATIONS FOR MEASURE DEVELOPMENT Gaps in measurement include assessment of behavioral and mental health and direct reports of care from the adolescent population. In the case of behavioral health and inpatient care, relevant CAHPS instruments exist for use in the adult population. We recommended that these be used as a basis for creating pediatric instruments for assessing care in these areas. For reports of care from the adolescent population, we encouraged states to consider use of the YAHCS to supplement their assessments from CAHPS instruments. States wanting more specific measures of clinical and patient-centered care and preventive and developmental services are encouraged to use the PHDS to supplement information garnered from CAHPS instruments. Another consideration for measure development is inpatient care. Although the NRC Picker instrument has good validity and reliability and has been used extensively by many institutions since its availability, one of its limitations is that its development and ownership is in the private sector. Thus, the ability to refine the survey is at the discretion of the proprietary owner. Adding supplemental items to assess specific areas would be difficult, and studies using data from a proprietary instrument such as the NRC Picker have not been extensive. Mechanisms need be developed to make proprietary instruments such as the NRC Picker more accessible for research, or developing a publically accessible instrument such as a pediatric version of the adult CAHPS Hospital Survey should be considered. The adults’ Hospital CAHPS has been endorsed by NQF, showing high construct validity and internal consistency
MEASURING FAMILY EXPERIENCES OF CARE
S65
and reliability.57 Modification of this instrument for use in children should be highly feasible given the CAHPS team’s prior experiences modifying adult instruments for use in the pediatric population. Development of a pediatric version of Hospital CAHPS also would facilitate an institution’s ability to benchmark pediatric services within a hospital with those that provide adult care. We also recommended further work be done to find ways to increase response rate and lower costs for receiving consumer feedback. Current response rates in the Medicaid population are low, raising concerns of sample bias. Both the CAHPS team and CAHMI have used the Internet for survey completion, and this and other modalities that could improve response merit further investigation. Finally, measurement developers should consider how existing patient/family experience measures developed for other purposes (ie, Patients Perception of Primary Care58 and Barriers to Care59) such as research can and should be used for assessing and improving quality of care. PATIENT AND FAMILY EXPERIENCE MEASURES RECOMMENDED BY SNAC The SNAC recommended core measurement set that went on for consideration by the Secretary of Health and Human Services included all of the measures recommended in this report. The final set, published on January 1, 2010, for public comment, included all these measures with the exception of the CAHPS Child Clinician & Group Survey. This measure was not selected because field experience with this survey was limited, with concerns about feasibility of implementation. Further experience with both the child and adult instruments will help in assessing the feasibility of more widespread use.60
ACKNOWLEDGMENTS Funding for this report was provided through purchase order HHSP233200900801P by the US Department of Health and Human Services Agency for Healthcare Research and Quality, with funds provided by its Centers for Medicare & Medicaid Services.
REFERENCES 1. Institute of Medicine. Envisioning the National Health Quality Report. Washington, DC: National Academies Press; 2001. 2. Institute of Medicine. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academies Press; 2001. 3. Buehler DM, Als H, Duffy FH, et al. Effectiveness of individualized developmental care for low-risk preterm infants: behavioral and electrophysiologic evidence. Pediatrics. 1995;96:924–932. 4. Clark NM, Gong M, Schork MA, et al. Impact of education for physicians on patient outcomes. Pediatrics. 1998;101:831–836. 5. Liptak GS. Enhancing patient compliance in pediatrics. Pediatrics. 1996;17:128–134. 6. Kaplan SH, Greenfield S, Ware JE Jr. Assessing the effects of physician-patient interactions on the outcomes of chronic disease. Med Care. 1989;27(suppl 3):S110–S127. 7. Institute of Medicine. Unequal Treatment: Confronting Racial and Ethnic Disparities in Health Care. Washington, DC: National Academies Press; 2001.
S66
CO ET AL
8. Zolnierek KB, Dimatteo MR. Physician communication and patient adherence to treatment: a meta-analysis. Med Care. 2009;47:826–834. 9. Harris-Kojetin LD, Fowler FJ Jr, Brown JA, et al. The use of cognitive testing to develop and evaluate CAHPS 1.0 core survey items. Consumer Assessment of Health Plans Study. Med Care. 1999;37(suppl 3):MS10–MS21. 10. Hays RD, Shaul JA, Williams VS, et al. Psychometric properties of the CAHPS 1.0 survey measures. Consumer Assessment of Health Plans Study. Med Care. 1999;37(suppl 3):MS22–MS31. 11. Hargraves JL, Hays RD, Cleary PD. Psychometric properties of the Consumer Assessment of Health Plans Study (CAHPS) 2.0 adult core survey. Health Serv Res. 2003;38(6 pt 1):1509–1527. 12. Shaul JA, Fowler FJ, Zaslavsky AM, et al. The impact of having parents report about both their own and their children’s experiences with health insurance plans. Med Care. 1999;37(suppl 3): MS59–MS68. 13. Rifkin L, Wolf MH, Lewis CC, Pantell RH. Children’s perceptions of physicians and medical care: two measures. J Pediatr Psychol. 1988; 13:247–254. 14. Schwab ME, Stone K. Conceptual and methodological issues in the evaluation of children’s satisfaction with their mental health care. Eval Program Plann. 1983;6:283–289. 15. Korsch BM, Gozzi EK, Francis V. Gaps in doctor-patient communication: doctor-patient interaction and patient satisfaction. Pediatrics. 1968;42:855–871. 16. Stewart TJ, Pantell RH, Dias JK, et al. Children as patients: a communications process study in family practice. J Fam Pract. 1981;13: 827–835. 17. Pantell RH, Stewart TJ, Dias JK, et al. Physician communication with children and parents. Pediatrics. 1982;70:396–402. 18. Homer CJ, Fowler FJ Jr, Gallagher PM, et al. The Consumer Assessment of Health Plan Study (CAHPS) survey of children’s health care. Jt Comm J Qual Improv. 1999;25:369–377. 19. Hedberg VA, Klein JD, Andresen E. Health counseling in adolescent preventive visits: effectiveness, current practices, and quality measurement. J Adolesc Health. 1998;23:344–353. 20. Bethell C, Klein J, Peck C. Assessing health system provision of adolescent preventive services: the Young Adult Health Care Survey. Med Care. 2001;39:478–490. 21. Santelli J, Klein J, Graff C, et al. Reliability of adolescent reporting of clinician counseling, health care use, and health behaviors. Med Care. 2002;40:26–37. 22. Gallagher PM, Fowler FJ, Elliott D. Collecting information about the health experiences of publicly insured adolescents. In: Cynamon ML, Kulka RA, eds. Seventh Conference on Health Survey Research Methods. Hyattsville, Md: National Center for Health Statistics; 1999:3–9. DHHS Publication No. (PHS) 01–1013. 23. Gallagher PM, Fowler FJ Jr. Collecting information about the health care experiences of adolescents. In: Proceedings of the Survey Research Methods Section, 1998. Alexandria, Va: American Statistical Association; 1998. 878–882. 24. Forrest CB, Simpson L, Clancy C. Child health services research. Challenges and opportunities. JAMA. 1997;277:1787–1793. 25. Schor EL. Use of health care services by children and diagnoses received during presumably stressful life transitions. Pediatrics. 1986;77:834–841. 26. General Accounting Office. Infants and Toddlers: Dramatic Increases in Numbers Living in Poverty. Washington, DC: General Accounting Office. GAO Pub No GAU/HEHS-94–74. Available at: http://archive. gao.gov/t2pbat3/151302.pdf; 1994. Accessed October 22, 2010. 27. US Department of Commerce, Bureau of the Census. Poverty in the United States, 1992. Current Population Reports. Washington, DC: US Department of Commerce, Bureau of the Census; 1993. 28. Chevarley FM, Owens PL, Zodet MW, et al. Health care for children and youth in the United States: annual report on patterns of coverage, utilization, quality, and expenditures by a county level of urban influence. Ambul Pediatr. 2006;6:241–264. 29. Homer CJ, Marino B, Cleary PD, et al. Quality of care at a children’s hospital. Arch Pediatr Adolesc Med. 1999;153:1123–1129.
ACADEMIC PEDIATRICS 30. Co JP, Ferris TG, Marino BL, et al. Are hospital characteristics associated with parental views of pediatric inpatient care quality? Pediatrics. 2003;111:308–314. 31. Co JP, Macdonald E, Yucel RM, Ferris TG. Practice variation in parental assessment of pediatric ambulatory care. Acad Pediatr. 2009;9:47–52. 32. Cleary PD. The increasing importance of patient surveys. Br Med J. 1999;319:720–721. 33. Bethell C. Patient-Centered Care Measures for the National Healthcare Quality Report. Portland, Ore: Foundation for Accountability (FACCT). Available at: http://www.policyarchive.org/handle/10207/ bitstreams/95530.pdf. Accessed October 22, 2010. 34. Solomon LS, Zaslavsky AM, Landon BE, Cleary PD. Variation in patient-reported quality among health care organizations. Health Care Financ Rev. 2002;23(4):85–100. 35. Nelson EC, Gentry MA, Mook KH, et al. How many patients are needed to provide reliable evaluations of individual clinicians? Med Care. 2004;42:259–266. 36. Fremont AM, Cleary PD, Hargraves JL, et al. Patient-centered processes of care and long-term outcomes of myocardial infarction. J Gen Intern Med. 2001;16:800–808. 37. Schneider EC, Zaslavsky AM, Landon BE, et al. National quality monitoring of Medicare health plans: the relationship between enrollees’ reports and the quality of clinical care. Med Care. 2001;39:1313–1325. 38. Chang JT, Hays RD, Shekelle PG, et al. Patients’ global ratings of their health care are not associated with the technical quality of their care. Ann Intern Med. 2006;144:665–672. 39. Rao M, Clarke A, Sanderson C, Hammersley R. Patients’ own assessments of quality of primary care compared with objective records based measures of technical quality of care: cross sectional study. Br Med J. 2006;333:19. 40. US Dept of Health and Human Services. CAHPS overview. Available at: https://www.cahps.ahrq.gov/content/cahpsOverview/OVER_ Intro.asp?p¼101&s¼1. Accessed October 22, 2010. 41. Fowler FJ Jr, Gallagher PM, Nederend S. Comparing telephone and mail responses to the CAHPS survey instrument. Med Care. 1999; 37:MS41–MS49. 42. The Child and Adolescent Health Measurement Initiative. Children with special health care needs: overview. Available at: http://cahmi. org/pages/Sections.aspx?section¼10. Accessed October 22, 2010. 43. US Dept of Health and Human Services. CAHPS Health Plan Survey and Reporting Kit 2008. Available at: https://www.cahps. ahrq.gov/CAHPSkit/Healthplan/HPChooseQx2.asp. Accessed October 22, 2010. 44. Gallagher P, Ding L, Ham HP, et al. Development of a new patientbased measure of pediatric ambulatory care. Pediatrics. 2009;124: 1348–1354. 45. Solomon LS, Hays RD, Zaslavsky AM, et al. Psychometric properties of a group-level Consumer Assessment of Health Plans Study (CAHPS) instrument. Med Care. 2005;43:53–60. 46. Solomon LS, Zaslavsky AM, Landon BE, Cleary PD. Variation in patient-reported quality among health care organizations. Health Care Financ Rev. 2002;23:85–100. 47. CAHPS Clinician and Group Survey and Reporting Kit 2008, Fielding the Survey. Available at: https://www.cahps.ahrq.gov/CAHPSkit/ Healthplan/HPChooseQx2.asp. Accessed October 22, 2010. 48. Bethell C, Peck C, Schor E. Assessing health system provision of well-child care: the Promoting Healthy Development Survey. Pediatrics. 2001;107:1084–1094. 49. Institute of Medicine. Preventing Mental, Emotional, and Behavioral Disorders Among Young People: Progress and Possibilities. Washington, DC: National Academies Press; 2009. 50. Eselius LL, Zaslavsky AM, Huskamp HA, et al. Case-mix adjustment of consumer reports about managed behavioral health care and health plans. Health Serv Res. 2008;43:2014–2032. 51. Eisen SV, Shaul JA, Clarridge B, et al. Development of a consumer survey for behavioral health services. Psychiatr Serv. 1999;50:793–798. 52. Shaul JA, Eisen SV, Stringfellow VL, et al. Use of consumer ratings for quality improvement in behavioral health insurance plans. Jt Comm J Qual Improv. 2001;27:216–229.
ACADEMIC PEDIATRICS 53. Eisen SV, Shaul JA, Leff HS, et al. Toward a national consumer survey: evaluation of the CABHS and MHSIP instruments. J Behav Health Serv Res. 2001;28:347–369. 54. Brown JA, Nederend SE, Hays RD, et al. Special issues in assessing care of Medicaid recipients. Med Care. 1999;37(suppl 3):MS79–MS88. 55. Dillman D, Sinclair M, Clark J. Effects of questionnaire length, respondent-friendly design, and a difficult question on response rates for occupant-addressed census mail surveys. Public Opin Q. 1993;57:289. 56. Safran DG, Karp M, Coltin K, et al. Measuring patients’ experiences with individual primary care physicians. Results of a statewide demonstration project. J Gen Intern Med. 2006;21:13–21.
MEASURING FAMILY EXPERIENCES OF CARE
S67
57. CAHPS Hospital Survey. Available at: http://hcahpsonline.org. Accessed October 22, 2010. 58. Seid M, Stevens GD, Varni JW. Parents’ perceptions of pediatric primary care quality: effects of race/ethnicity, language, and access. Health Serv Res. 2003;38:1009–1031. 59. Sobo EJ, Seid M, Reyes Gelhard L. Parent-identified barriers to pediatric health care: a process-oriented model. Health Serv Res. 2006;41:148–172. 60. 74 Federal Register 68846–68849 (2009). Available at: http://www. gpo.gov/fdsys/pkg/FR-2009–12–29/html/E9–30802.htm. Accessed November 9, 2010.