- Email: [email protected]
The ghost in the machine in health reform
POLICY WATCH
tify problems reliably or are they potential sheep in wolves’ clothing? It would seemto the casual observer that both the Health Care Financing Administration and those who are the targets of the evaluation processwould be interested in the answer to this question. In a recent article, Rubin et al reviewed the literature and available data on the PROS and came to the conclusion that the PROS’ initial screeningprocess,the PRO physician review process,andthe reviewprocessas a whole were not detecting as many problems asthey should.It would appearthat there hasbeen far too little evaluation done on the PRO processand little in the way of empirical demonstrations of the effectivenessof the approach. To further evaluate their conclusions,the authors conducteda retrospectivestudy of one state’s PRO review process (including the initial screening,the physician review, and the final judgments). They evaluated the PRO’s effectivenessby comparing its judgments with an independent study of a sample of the charts from the original PRO review. The independent study judgment wasbasedon a blinded, structured but implicit review of a sample of hospital records in the state, which served as the gold standard against which the original review would be judged. The key notions about the study group are: unlike the PRO’s reviews,the study’s evaluatorswere blinded and useda structured review processbut still retained the implicit judgments as in the case of most PRO reviews. Thus the study review should be expected to bemore reliable (reproducible) and perhaps more valid (on target). The study coveredthe period from 1985to 1987. What did they find? In two words, little agreement! However,let us focus on three points.
The PRO initial screen found only one of three recordsthat the study discovered were below standard.Amazingly, only one of three charts found substandard by the PRO physicians were in agreement with the study’s judges! Even the direction of the differences was dissimilar. The final PRO quality of care judgments and the study judgments agreed only slightly more than would be expectedby relying on chance (flipping 25-cent pieces would have been cheaper)or as the authors note: the screening processwasonly slightly better at identifying substandard care than had the charts been randomly selected. The authors discussthe problems and possible solutions to these shortcomings. Several of their suggestionsto improve the physician review component of the PRO review processwould be well known to individuals working in the areaof the assessment of clinical competence and are worth repeating: structure helps the reliability of a reviewprocess; careful selection and training of judges and examiners are necessary; and where soft judgments are being made without preset criteria, more judges help. In short, in this example,there is no where to go but up by getting back to basics.How about PROS for the PROS?Maybe not on second thought, but a few goodexperts on clinical assessment would do wondersto the PROS’ “hit-rates.” What doesthis study mean for the typical physician? For starters,the professionmust be better at and more awareof the preventive actionssuch as communicating better with the patient and documenting everything in the chart, including the patient’s dischargeinstructions. While this is only one study of one state’s PRO, the results suggest that physicians and hospitals should
monitor their local PROS and considercollaboratingwith them to develop better feedback systems for educatingproviders. As the authors note, the use of multiple physician reviewers is not fiscally feasible, so other means of improving the processlike better training and better review tools are needed.In either case, more studiesof this nature areessential if the PRO processis to be both effective and creditable.WDD
The Ghost in the Machine in Health Reform [Wilkinson RG. National mortality rates: the impact of inequality? Am J Public Health 1992; 82: 1082-4.1 [Keil JE, Sutherland SE, Knapp RG, Tyroler HA. Does equal socioeconomic status in black and white men mean equal risk of mortality? Am J Public Health 1992; 82: 11334
H
ealth care reform in the United Statesis predicated on changesin health services financing so the uninsured millions can have ready accessto care. Underpinning this paradigm is the assumption that improving accessto basichealth services will demonstrably improve the health status of the nation. Thesetwo articles suggesta ghost in the machine that arguably threatens the momentum of reform. Wilkinson reviews worldwide statistics on the relationship between mortality and socioeconomic status.For peopleliving in developedcountries,overall mortality rates have not improved despite continued increases in standardof living asmeasuredby per capita income. He concludes that in developednations,it is incomedistribution rather than per
December 1992 The American Journal of Medicine
Volume 93
III
POLICY WATCH
capita income that best correlates with mortality statistics. On the other hand, in the less developed nations, mortality is more associated with improvements in per capita income up to a threshold of about $5,000 (by 1985 U.S. dollars). Therefore, if the developed nations are to achieve further improvements in life expectancy, they must focus on shrinking the income distribution differentials between socioeconomic groups. For example, Japan and Great Britain were comparable in terms of income distribution and life expectancy in 1970. While Japan’s income distribution has narrowed to become the “narrowest of any recorded in the United Nations Human Development Report” and life expectancy has increased to unprecedented levels, British income distribution has widened and life expectancy has slipped compared to other nations. These data suggest that income distribution has an effect
on mortality through the impact of relative income. Additionally, this effect on health status highlights the intimacy health care reform shares with other social reforms that would stimulate economic growth. Health reform in an economic milieu that promotes growth while preserving income differentials may do little to improve national health statistics. Wilkinson also comments that the racial differences in health status found in this country reflect poverty and deprivation, not some complex interaction of biology and risk factors. Keil’s work with the Charleston Heart Study supports Wilkinson’s observations via-h-via race and health status. Blackwhite differences in cardiovascular mortality disappear when socioeconomic status is controlled. In fact, black mortality experience is slightly better than among whites in comparable socioeconomic strata. The authors recall Antonosky’s admonition in the 1960s: “The inescapable conclu-
sion is that social class influences one’s chance of staying alive.” The body of evidence linking social class to health status is increasingly powerful and compelling and poses a dilemma for health care reformers: whether to agitate for universal access to care through changes in the financing of health care first and then address the problems of income distribution, or vice versa? Fixing the financing of the health care system becomes a piece of cake when faced with the deeply rooted economic problems facing this nation. Health reform as it is currently being formulated may make a lot of folks feel better and will definitely make more providers wealthier, but will probably do little to improve our ratings on the Top 10 International Health Rankings. Lest we set ourselves up for failure, we should see the “ghost in the machine” so we are realistic about our expectations for reform in a socioeconomically distorted society.-MM
POLICY SPEAK
I
“The universal access single-payer health plan would not automatically ensure highquality care or lead to improved information about efficacy of various medical interventions. Achieving this goal under any system hinges on expansion of publicly-sponsored effectiveness research. lf the dollar limit used to cap the liability of employer-sponsored health plans for acute care were raised annually by the rate at which health care spending rose, private insurers would continue to compete for sales on the basis of service, quality monitoring, and benefits other than those required under law.”
IV
December 1992 The American Journal of Medicine
Volume 93
-Aaron
HJ. Choosing from the health care reform menu. J Am Health Policy 1991; 1: 27.
“Policymakers took a naive ‘deregulate and forget’ approach to competitive reform during the 198Os, that never can succeed. Any future attempt to use market forces to reform the health care system must distinguish between socially desirable and undesirable forms of competition and incorporate policies that encourage the former while deterring the latter.”
-Higgins W. Dominos that wouldn’t fall: understanding the failure of competitive reform. J Am Health Policy 1991; 1: 33.
tify problems reliably or are they potential sheep in wolves’ clothing? It would seemto the casual observer that both the Health Care Financing Administration and those who are the targets of the evaluation processwould be interested in the answer to this question. In a recent article, Rubin et al reviewed the literature and available data on the PROS and came to the conclusion that the PROS’ initial screeningprocess,the PRO physician review process,andthe reviewprocessas a whole were not detecting as many problems asthey should.It would appearthat there hasbeen far too little evaluation done on the PRO processand little in the way of empirical demonstrations of the effectivenessof the approach. To further evaluate their conclusions,the authors conducteda retrospectivestudy of one state’s PRO review process (including the initial screening,the physician review, and the final judgments). They evaluated the PRO’s effectivenessby comparing its judgments with an independent study of a sample of the charts from the original PRO review. The independent study judgment wasbasedon a blinded, structured but implicit review of a sample of hospital records in the state, which served as the gold standard against which the original review would be judged. The key notions about the study group are: unlike the PRO’s reviews,the study’s evaluatorswere blinded and useda structured review processbut still retained the implicit judgments as in the case of most PRO reviews. Thus the study review should be expected to bemore reliable (reproducible) and perhaps more valid (on target). The study coveredthe period from 1985to 1987. What did they find? In two words, little agreement! However,let us focus on three points.
The PRO initial screen found only one of three recordsthat the study discovered were below standard.Amazingly, only one of three charts found substandard by the PRO physicians were in agreement with the study’s judges! Even the direction of the differences was dissimilar. The final PRO quality of care judgments and the study judgments agreed only slightly more than would be expectedby relying on chance (flipping 25-cent pieces would have been cheaper)or as the authors note: the screening processwasonly slightly better at identifying substandard care than had the charts been randomly selected. The authors discussthe problems and possible solutions to these shortcomings. Several of their suggestionsto improve the physician review component of the PRO review processwould be well known to individuals working in the areaof the assessment of clinical competence and are worth repeating: structure helps the reliability of a reviewprocess; careful selection and training of judges and examiners are necessary; and where soft judgments are being made without preset criteria, more judges help. In short, in this example,there is no where to go but up by getting back to basics.How about PROS for the PROS?Maybe not on second thought, but a few goodexperts on clinical assessment would do wondersto the PROS’ “hit-rates.” What doesthis study mean for the typical physician? For starters,the professionmust be better at and more awareof the preventive actionssuch as communicating better with the patient and documenting everything in the chart, including the patient’s dischargeinstructions. While this is only one study of one state’s PRO, the results suggest that physicians and hospitals should
monitor their local PROS and considercollaboratingwith them to develop better feedback systems for educatingproviders. As the authors note, the use of multiple physician reviewers is not fiscally feasible, so other means of improving the processlike better training and better review tools are needed.In either case, more studiesof this nature areessential if the PRO processis to be both effective and creditable.WDD
The Ghost in the Machine in Health Reform [Wilkinson RG. National mortality rates: the impact of inequality? Am J Public Health 1992; 82: 1082-4.1 [Keil JE, Sutherland SE, Knapp RG, Tyroler HA. Does equal socioeconomic status in black and white men mean equal risk of mortality? Am J Public Health 1992; 82: 11334
H
ealth care reform in the United Statesis predicated on changesin health services financing so the uninsured millions can have ready accessto care. Underpinning this paradigm is the assumption that improving accessto basichealth services will demonstrably improve the health status of the nation. Thesetwo articles suggesta ghost in the machine that arguably threatens the momentum of reform. Wilkinson reviews worldwide statistics on the relationship between mortality and socioeconomic status.For peopleliving in developedcountries,overall mortality rates have not improved despite continued increases in standardof living asmeasuredby per capita income. He concludes that in developednations,it is incomedistribution rather than per
December 1992 The American Journal of Medicine
Volume 93
III
POLICY WATCH
capita income that best correlates with mortality statistics. On the other hand, in the less developed nations, mortality is more associated with improvements in per capita income up to a threshold of about $5,000 (by 1985 U.S. dollars). Therefore, if the developed nations are to achieve further improvements in life expectancy, they must focus on shrinking the income distribution differentials between socioeconomic groups. For example, Japan and Great Britain were comparable in terms of income distribution and life expectancy in 1970. While Japan’s income distribution has narrowed to become the “narrowest of any recorded in the United Nations Human Development Report” and life expectancy has increased to unprecedented levels, British income distribution has widened and life expectancy has slipped compared to other nations. These data suggest that income distribution has an effect
on mortality through the impact of relative income. Additionally, this effect on health status highlights the intimacy health care reform shares with other social reforms that would stimulate economic growth. Health reform in an economic milieu that promotes growth while preserving income differentials may do little to improve national health statistics. Wilkinson also comments that the racial differences in health status found in this country reflect poverty and deprivation, not some complex interaction of biology and risk factors. Keil’s work with the Charleston Heart Study supports Wilkinson’s observations via-h-via race and health status. Blackwhite differences in cardiovascular mortality disappear when socioeconomic status is controlled. In fact, black mortality experience is slightly better than among whites in comparable socioeconomic strata. The authors recall Antonosky’s admonition in the 1960s: “The inescapable conclu-
sion is that social class influences one’s chance of staying alive.” The body of evidence linking social class to health status is increasingly powerful and compelling and poses a dilemma for health care reformers: whether to agitate for universal access to care through changes in the financing of health care first and then address the problems of income distribution, or vice versa? Fixing the financing of the health care system becomes a piece of cake when faced with the deeply rooted economic problems facing this nation. Health reform as it is currently being formulated may make a lot of folks feel better and will definitely make more providers wealthier, but will probably do little to improve our ratings on the Top 10 International Health Rankings. Lest we set ourselves up for failure, we should see the “ghost in the machine” so we are realistic about our expectations for reform in a socioeconomically distorted society.-MM
POLICY SPEAK
I
“The universal access single-payer health plan would not automatically ensure highquality care or lead to improved information about efficacy of various medical interventions. Achieving this goal under any system hinges on expansion of publicly-sponsored effectiveness research. lf the dollar limit used to cap the liability of employer-sponsored health plans for acute care were raised annually by the rate at which health care spending rose, private insurers would continue to compete for sales on the basis of service, quality monitoring, and benefits other than those required under law.”
IV
December 1992 The American Journal of Medicine
Volume 93
-Aaron
HJ. Choosing from the health care reform menu. J Am Health Policy 1991; 1: 27.
“Policymakers took a naive ‘deregulate and forget’ approach to competitive reform during the 198Os, that never can succeed. Any future attempt to use market forces to reform the health care system must distinguish between socially desirable and undesirable forms of competition and incorporate policies that encourage the former while deterring the latter.”
-Higgins W. Dominos that wouldn’t fall: understanding the failure of competitive reform. J Am Health Policy 1991; 1: 33.