Policy analysis in an information-rich environment

ARTICLE IN PRESS

Social Science & Medicine 58 (2004) 2231–2241

Policy analysis in an information-rich environment Leslie L. Roos*, Verena Menec, R.J. Currie Manitoba Centre for Health Policy, Department of Community Health Sciences, Faculty of Medicine, University of Manitoba, 408-727 McDermot Avenue, Winnipeg, MB, Canada R3E 3P5

Abstract Population-based insurance systems using longitudinal administrative data and record linkage techniques have helped create ‘‘information-rich’’ environments in several sites around the world. The output of five research groups using administrative data (Oxford, Western Australia, and three Canadian centres: Manitoba, Ontario and British Columbia) was analysed from contacts with the research groups and through use of the National Library of Medicine’s PubMed and Medical Subject Headings (MeSH) categories. MeSH words ‘‘utilization’’, ‘‘economics’’, ‘‘physicians’’, and ‘‘physician practice patterns’’ more frequently characterized the research by the three Canadian centres than that of the other sites. With core funding for deliverables negotiated with the provincial health ministries, Canadian researchers have been more likely to use linked databases for policy analyses. Manitoba examples highlight the capabilities associated with these information-rich environments. They include the ability to analyse interventions longitudinally; to compare regions, areas and hospitals in defined populations; to combine information on patients and physicians; to add up expenditures for different services within the Canadian health-care system; and to examine population health issues in areas such as education and family services. Wellorganized data and the capability for rapid response have been critical for timely policy analysis in Manitoba. A number of successes are mentioned; less successful efforts to influence practice patterns and to modify the internal workings of hospitals are noted. Investments in filling gaps in data collection and in enriching existing data would facilitate additional research. Planning and managing health care for an entire population has benefited greatly from the development of an information-rich environment. r 2003 Elsevier Ltd. All rights reserved. Keywords: Health services research; Policy analysis; Longitudinal; Population based; Health information systems; Canada

Introduction Population-based insurance systems using longitudinal administrative data and techniques linking records across multiple files have helped create ‘‘informationrich’’ environments in several sites around the world. Working in Western Australia to develop a linked database (Holman, Bass, Rouse, & Hobbs, 1999) identified the Oxford Record Linkage Study, the *Corresponding author. Manitoba Centre for Health Policy, Department of Community Health Sciences, Faculty of Medicine, University of Manitoba, 408-727 McDermot Avenue, Winnipeg, MB, Canada R3E 3P5. Tel.: 204-789-3773; fax: 204-789-3910. E-mail address: leslie [email protected] (L.L. Roos).

Scottish Record Linkage System, the Rochester Epidemiology Project, and the Manitoba Centre for Health Policy as ‘‘centres known internationally for the conduct of research using record linkage.’’ Two other Canadian groups, the Centre for Health Services and Policy Research in British Columbia and the Institute for Clinical and Evaluative Sciences in Ontario, have also been using administrative data for both provincially funded policy work and investigator-initiated research. Routine updates of administrative data (building on consistent formats) facilitate the study of change; analyses of variation in population-based rates over time and cohort studies are often performed (Goldacre, Griffith, Gill, & Mackintosh, 2002; Kendrick, Douglas, Gardner, & Hucker, 1998; Melton, 1996).

0277-9536/$ - see front matter r 2003 Elsevier Ltd. All rights reserved. doi:10.1016/j.socscimed.2003.08.008

ARTICLE IN PRESS 2232

L.L. Roos et al. / Social Science & Medicine 58 (2004) 2231–2241

Lists of publications, projects, and investigators were obtained for five centres: Oxford, Western Australia, Manitoba, British Columbia, and Ontario. Some research groups (such as in Manitoba) concentrated more on administrative data than others. Substantial efforts were made to identify all relevant publications. Each publication with a title suggesting the use of administrative data was looked up using the National Library of Medicine’s PubMed. Abstracts were read by a research associate and by the senior author (LLR) to make sure that administrative data were used. Common exclusion rules were applied across the centres: only studies published from 1994 to 2002 were included. Commentaries and letters were excluded, as was work based on a single disease or surgical registry. Several Canadian studies included authors from more than one centre; the centre providing the data was credited for the research. In multi-centre studies, all participating centres were credited. Each relevant abstract and attached Medical Subject Headings (MeSH) categories were entered into a centre-specific Reference Manager database. MeSH categories were analysed using SAS version 8.2. Despite PubMed’s independent coding and economy, important research might be missed for several reasons: because the work is never published in an academic format (a problem with many projects conducted for government agencies) and because book chapters and certain journals (such as the Canadian Journal on Aging) are not included in PubMed. Further details on the methodology are available from the senior author. The MeSH word ‘‘epidemiology’’ was most used in reference to the Oxford and Western Australia centres. The terms ‘‘utilization’’, ‘‘economics’’, ‘‘socioeconomic factors’’, ‘‘physicians’’, and ‘‘physician practice patterns’’ were more frequently used to characterize the research of the three Canadian groups than that of the other sites. Canadian research centres oriented towards local policy problems receive much of their funding from provincial Ministries of Health. This thrust seems to have pushed the Canadian centres in roughly similar directions. Studies of specific diseases and diagnoses have been popular among all centres; the coding for ‘‘disease’’ in Oxford and ‘‘incidence’’ in Western Australia appears to reflect a broad range of outcomes research at those sites (Table 1). ‘‘Utilization’’ and ‘‘socioeconomic factors’’ were particularly frequent for Manitoba. Published studies with a defined geographic focus, those involving rural populations and others analysing urban populations, were noted more often in Manitoba than elsewhere. The relatively low frequency of ‘‘cohort studies’’ and ‘‘case control’’ in Manitoba reflects less of an emphasis on clinical epidemiology and a greater concern with policy analysis. The Manitoba Centre has negotiated a series of annual deliverables; almost all of the projects explicitly

take advantage of the available databases. As categorized by Manitoba Health, the great majority of projects over the last 13 years have focused on: health and quality of health care (9), regional health authorities— both urban (14) and rural/northern (5), physicians and human resources (5), and financial and cost issues (8). One set of deliverables (11) concerned development of the database and information system. These general topics have all been noted as of national concern (Iglehart, 2000). In Manitoba the capabilities for policy analysis include the ability to: (1) (2) (3) (4)

study interventions longitudinally, compare regions, areas, and hospitals, combine information on patients and physicians, add up expenditures for different services within the Canadian health-care system, and (5) examine the determinants of health using education and family services data in conjunction with healthrelated information. Given the great interest in health policy among OECD countries, these information-rich environments should be more extensively used for policy analysis. This paper provides examples from recent Manitoba work, highlighting successful uses of the database which build on its special capabilities. Limitations—both in terms of less successful efforts to influence policy and gaps in available data—are also discussed.

Data The inner ring of Fig. 1 outlines the Manitoba database with a population-based research registry playing a central role. Besides the three provinces noted earlier, several other Canadian provinces (Saskatchewan, Nova Scotia, Quebec) have organized data in a similar fashion. Longitudinal or linked data are typically put together as needed for each study. The research registry developed by the Manitoba Centre for Health Policy has integrated information from the Manitoba Health registry and other sources in a convenient format. Specifically, the research registry contains an encrypted number assigned to each resident, demographic characteristics, place of residence (a 6-digit postal code), and family composition. Population-based registries, building on Vital Statistics files, can provide information on all residents in an area, as well as their dates of arrival and departure (births, deaths, and moves) for any date since 1970 (Roos & Nicol, 1999). Time-sensitive data elements (place of residence, family composition) are updated using ‘‘snapshot’’ registries provided every six months. Each substantive file included in the inner ring of studies can be checked

ARTICLE IN PRESS L.L. Roos et al. / Social Science & Medicine 58 (2004) 2231–2241

2233

Table 1 Study content across five research centers using administrative data MeSH words

Oxford (%)

Western Australia (%)

Canada Manitoba (%)

Epidemiology Health Services Research Disease Incidence Diagnosis Child Adolescent Adult Middle Age Aged Utilization Mortality Cohort Studies Case Control Type of Study: Comparative Retrospective Economics Populations: Rural Urban Socioeconomic Factors Physicians Physician’s Practice Patterns Number of Papers Published

Ontario (%)

British Columbia (%)

83 10

78 3

47 23

42 6

61 10

22 17 12 20 27 51 49 46 15 41 5 29

0 32 21 24 22 46 52 59 10 41 13 3

0 8 14 34 36 51 42 41 51 28 7 2

0 7 20 7 11 33 47 80 29 38 27 2

0 13 13 16 16 29 35 61 42 26 32 6

15 15 5

27 10 8

24 5 18

22 29 13

19 13 29

0 0 2

0 2 5

8 9 22

1 1 3

0 0 6

0

0

16

16

0

0

0

5

19

6

41

63

146

138

31

Papers published between 1994 and 2002 using administrative data were counted using Pub Med’s MeSH (Medical Subjects Headings) words.

against the registry for accuracy of the identifiers and particular information (for example, date of in-hospital death) (Roos, Soodeen, Bond, & Burchill, 2003). Such databases can be appropriately aggregated at the level of the individual, the physician, the hospital, the region, or the population as a whole. The outer ring of studies in Fig. 1 represents special files—both clinical and survey data—generated for various Manitoba projects (Kryger, Walld, & Manfreda, 2002; Mustard, Derksen, Berthelot, & Wolfson, 1999). Similar files, linked to administrative data, have been used in other centres, both in Canada and elsewhere (Abbass, 2002; Rankin et al., 1999; Rosman, 2001). The linkages with the National Population Health Survey (and other Statistics Canada surveys) and the social and educational data recently provided by several Manitoba ministries are particularly valuable for studying the

determinants of health in a population (Kindig & Stoddart, 2003).

Results Longitudinal analyses Provincial health reform The evaluation of interventions such as those associated with health reform is of ongoing importance to Canadian provincial governments. Appropriate research designs use the strength of the available databases—longitudinal information on several types of utilization, measurable indicators of health, the availability of control groups, and so forth. Thus, Manitoba researchers examined the period from 1989

ARTICLE IN PRESS L.L. Roos et al. / Social Science & Medicine 58 (2004) 2231–2241

2234

Inflammatory Bowel Disease Database

Sleep Lab Clinical Data

Hospital

Diabetes Education Resource Database

Alcoholism Panel Surveys

Medical Nursing Home

Home Care Population - Based Research Registry

Cancer Registry Pharmaceutical

Provider

Immunization Monitoring

Heart Health Survey

Social & Educational Data

Vital Statistics

Aging in Manitoba Study

National Population Health Survey

Fig. 1. Administrative data in an information-rich environment.

(to provide several years of data before downsizing of the hospital system began in 1992) to 1998 to determine whether closures of acute care hospital beds adversely affected residents’ access to care, the quality of care delivered, or the health of the population (Brownell, Roos, & Roos, 2001). Access to hospital services was measured by the number of patients treated, the types and amount of services provided, length of stay, and location of the service (inpatient or outpatient). Readmission and mortality rates after discharge provided measures of quality of care, while the health of the population was assessed using mortality rates. Even with fewer resources, just as many patients were cared for in 1998 as before downsizing. The number of days spent in hospital decreased dramatically (by more than 19% for shortstay cases in 1997). A shift in surgery from inpatient to outpatient settings accounted for much of this decline. Utilization decreased less for the poor than for other groups, indicating that vulnerable populations were somewhat protected during the system change. Mortality rates within 30 days of discharge remained stable for heart attack patients and dropped for those undergoing surgery for hip fracture or cancer. Readmissions within 30 days of discharge, increased initially, but by 1997 readmissions in all but one (digestive disorders) of the 13 categories studied dropped to pre-downsizing rates. The effects of both health reforms and technological advances were further analysed for four common surgical procedures from 1991 to 1996. Cholecystectomy

(62%) and hernia repair (58%) showed the greatest declines in length of stay, which was in part due to changed technology (laparascopic surgery) and in part due to bed closures and a shift to outpatient surgery. Declines in length of stay generated by bed closures alone—those for hysterectomy (36%) and appendectomy (23%)—were smaller. Neither shorter stays nor the move to outpatient treatment was associated with more readmissions or greater mortality (Soodeen, Roos, & Peterson, 2000).

Overcrowding and ‘‘Hallway Medicine’’ One Manitoba study of severe seasonal hospital bed shortages gained considerable attention. During the winter of 1999–2000, the Canadian news media frequently described emergency rooms overrun with flu patients and put on ‘critical care bypass’, with victims of ‘‘hallway medicine’’ stranded on stretchers and surgeries being postponed. Seasonal patterns of use for Winnipeg’s seven acute care hospitals were tracked over the previous 11 years. Almost every winter, the number of patients arriving at hospital increased 10% above average for approximately 1 to 3 weeks. This occurred as far back as the late 1980s, prior to the loss of 700 beds as part of health reform, suggesting that bed availability was not the central cause. The main reasons for the increased pressure over the entire period were pneumonia, influenza, and other respiratory conditions often linked to influenza.

ARTICLE IN PRESS L.L. Roos et al. / Social Science & Medicine 58 (2004) 2231–2241

The Winnipeg Regional Health Authority adopted several recommendations for preventing hospital overcrowding: (1) implement a comprehensive campaign of flu vaccination, (2) increase capacity to discharge patients on weekends, (3) increase efficiency of patient transfer to personal care homes (Menec, Roos, & MacWilliam, 2002). Early planning and preventive measures were generally successful in Manitoba, Saskatchewan, and Alberta. In contrast, hospitals in Ontario that did not scale back on discretionary surgery or keep beds open during the holidays experienced many adverse effects including last minute cancellations for surgery and frustrated patients (Gray, 2000). With the Canadian system functioning with a tight bed supply, both a public health perspective (reduce the incidence of influenza and pneumonia through immunization) and a managerial perspective (improve admission and discharge practices) proved helpful. Comparing regions and areas Regional health authorities Health reform initiatives leading to decentralization and to the establishment of Regional Health Authorities (RHAs) in nine of the 10 Canadian provinces (Ontario being the exception) have led to an increased number of decision-makers seeking information. Although the powers allocated to the regions within each province vary across the country, the RHAs are usually responsible for hospitals, nursing homes, and home care services. With six Canadian provinces having populations around 1 million or less, these authorities are often quite small. Even in larger provinces, managers complain about the lack of timely, comparative data on population health and functioning of the health-care system (Casebeer & Johnson, 2000). Centralized data analysis appears cost effective; several European countries have found that ‘‘the lower down in the public sector they decentralize power over the health system, the more important it becomes to have a central structure to set standards, monitor and evaluate performance, and prevent opportunistic behaviors’’ (Saltman & Figueras, 1998). The Manitoba Centre for Health Policy has compared demographic, socioeconomic, health status, and health services indicators across nine rural and northern Health Authorities, providing profiles for individual RHAs and their districts. Detailed geographic coding associated with both place of residence and place of service delivery, as well as validated indicators of health, support these analyses (Robinson, Young, Roos, & Gelskey, 1997). Regional authorities want to be able to target programmes to particular areas within their regions. For example, work for the rural South Eastman RHA highlighted three immediate problems: the relatively poor health status among a specific at-risk group

2235

(residents of the RHA’s Southern District), physician maldistribution, and inconsistent patterns of service use (Roberts, Fransoo, Black, Roos, & Martens, 2002). Through the mandated planning process, the RHA received funding for more physician positions and additional primary health-care services. More generally, the information helps to track some of Canada’s more enduring concerns: physician distribution and vulnerable populations (Mustard, Derksen, & Black, 1999; Barer & Stoddart, 1992). Widening comparisons Combining information on all aboriginal persons having band membership in a First Nations community with geographic codes permitted analysis of health and health care for the entire Manitoba First Nations population. Considerable variation in life expectancy and disease prevalence was noted among different Tribal Councils as well as between First Nations members and all other Manitobans. Many Tribal Councils in generally healthy southern Manitoba were substantially sicker than their counterparts in the more isolated north. Such results, some of which were quite unexpected, provide a baseline for targetting programmes for improvement and evaluating these efforts (Martens et al., 2002). Finally, two different approaches generated performance indicators for Manitoba’s rural and northern hospitals (Martens, Stewart, Mitchell, & Black, 2002). Population-based indicators include, for example, the number of people served by the hospital(s) in each area, the neediness and health status of the population, and hospital use relative to need (Naylor & Slaughter, 1999; Wennberg & Cooper, 1999). More traditional hospitalbased indicators measure hospital activity according to such criteria as case intensity, resource use, and occupancy rates. The information has generated considerable interest from Manitoba’s Regional Health Authorities. If bed or hospital closures result, public complaints are likely. But outcomes need not suffer; both British Columbia and Saskatchewan showed no increase in mortality associated with a reduction in hospital beds (Liu, Hader, Brossart, White, & Lewis, 2001; Sheps et al., 2000). Indeed, life expectancy among residents of affected communities increased rather than decreased after bed closures in rural Saskatchewan. Physicians and patients Physician resource planning The capability of combining information on physicians and patients has supported research on two important issues: physician resource planning and primary health-care reform. The ability to create physician service areas by dividing Manitoba geographically on the basis of physician use patterns of area

ARTICLE IN PRESS 2236

L.L. Roos et al. / Social Science & Medicine 58 (2004) 2231–2241

residents is critical for such planning (Roos et al., 1999a). Key characteristics of these areas included: * * *

*

health of the population, socioeconomic risk characteristics of area residents, access to and utilization of physicians by area residents no matter where contact took place, and supply of physicians in the area.

These data have challenged beliefs about an increasing shortage of physicians in at least some surgical specialties. Debate about the implications of the findings of no overall crisis of physician supply, both in Manitoba and nationally has continued (Watson et al., 2003; Wharry & Sibbald, 2002). Primary health-care reform Registration of patients with a physician has been increasingly suggested as part of a reform of primary health care (Romanow, 2002). Such formal registration would allow ‘‘capitation’’, paying clinics a set amount per year per patients no matter how many services they provide (Reid et al., 2001). One Manitoba study noted substantial variability across practices in the proportion of informally registered patients (15–68% receiving most of their care from one practice) and the number of ‘informally registered’ patients per physician (544–1378 patients per full-time equivalent physician) (Menec, Black, Roos, & Bogdanovic, 2001). The average health of patients seen by one clinic varies widely from the average health of patients seen by another (Reid et al., 2001). The adjustment for differences in patient health needed for capitation payments would in themselves be controversial. Thus, the potential disruption to both patients and physicians in moving towards registration should not be underestimated. The relatively low levels of existing informal registration suggest a need to enhance access by, for example, providing after-hours services. Adding up expenditures Provincial costs Expanding the capabilities of the provincial data to permit costing studies has proved important both for Manitoba work and for national debates about healthcare financing. A methodology for assessing the costs of hospital care combined hospital-specific cost information included in reports to federal statistical agencies with case mix information for each provincial hospital (Shanahan, Loyd, Roos, & Brownell, 1999). Initial work calculated the cost of inpatient hospital care, compared the costs of different hospitals, and then estimated the cost of providing care to residents of the various regions. The basic split was between Winnipeg, the capital city with over half the provincial population, and the rest of Manitoba. Per capita expenditures for Winnipeg resi-

dents were only 6% higher than for non-Winnipeg residents. The largest disparity in spending was for physicians and other professionals; 33% more was spent for delivery of care to Winnipeg residents than to nonWinnipeg residents (Shanahan, Steinbach, Burchill, Friesen, & Black, 1999). A second project incorporated the case-mix-adjusted costing methodology to develop a list of costs for provincial inpatient services (Jacobs & Roos, 1999). This list used ‘‘standard’’ costs, applying the same unit costs to a given group of services, regardless of where provided. Such a cost list allows comparisons across many different settings and was expanded to provide national estimates for Canada (Jacobs et al., 2000). Medical Savings Accounts Medical Savings Accounts attempt to reduce healthcare costs by transferring responsibility for expenditures to patients, while providing them with state-supported base amounts to cover some of the costs (Forget, Deber, & Roos, 2002). Costs readily attributable to Manitoba individual residents for ambulatory physician visits and admissions to hospital during fiscal years 1997–1999 averaged $730 each year. About 58% of hospital costs and almost all costs (over 95%) associated with ambulatory physician visits were easily assignable to individuals (Watson et al., 2003). About 40% of Manitobans used less than $100 each, and 80% used less than $600. The highest-using 1% of the Manitoba population accounted for 26% of spending on hospital and physician care, whereas the lowest-using 50% accounted for just 4%. Age-specific results were similar; even most elderly individuals fall into the low-usage category. To assure policy relevance, assumptions about Medical Savings Accounts included a plausible annual entitlement (the current average cost of $730 per year), a reasonable catastrophic threshold (the threshold above which costs are paid by the government—$1000 per year), and that individuals spending less than their entitlement would be allowed to keep the surplus. The implementation of polices based on such assumptions would have led to a 54% increase in spending on hospital and physician costs easily allocatable to individuals. This increase was concentrated on the healthiest members of the population. Because two major government reports on the future of Canadian Medicare were published in 2002, a number of Canadian newspapers and at least one American commentator ran related stories (Canada Senate, 2002; Romanow, 2002). The Globe and Mail, arguably Canada’s premier newspaper, presented a news story, two Op-Ed pieces, and an editorial referring to the paper. Canadian proponents of Medical Savings Accounts responded to this analysis by suggesting plans with

ARTICLE IN PRESS L.L. Roos et al. / Social Science & Medicine 58 (2004) 2231–2241

differing assumptions regarding unused funds. With utilization and illness concentrated in a relatively small percentage of the population, Medical Savings Accounts are tempting for private insurers (as in the United States) who can target a disproportionately healthy group (Kobliner, 2003). However, a government insurer providing for an entire population must take a different view. Where little systematic information has been previously available, well-organized data and the capability for rapid analysis proved critical for timely work. Determinants of health Concerns about growing gaps between rich and poor have stimulated research on the broad determinants of health (Anderson, Hurst, Hussey, & Jee-Hughes, 2000; Link, Northridge, Phelan, & Ganz, 1998). Although international comparisons reflect well on Canadian life expectancy, among the key questions posed by the federally funded Canadian Population Health Initiative are: (1) Why are some communities healthy and others not? (2) To what extent do Canada’s major policies and programs improve population health? (3) How do social roles at work, in the family, and in the community affect health status over the life course? These questions are germane, given recent increases in premature mortality among Manitoba residents of the lowest income neighbourhoods (who already had the highest mortality rates) (Lix, Newburn-Cook, Roos, & Derksen, 2002). Linkage of administrative data with external data sets (as outlined in Fig. 1) can be most helpful for studying health determinants and health outcomes. Provinces have been encouraged to ‘‘buy’’ larger samples on Statistics Canada’s national surveys for their own purposes. Under stringently defined conditions, record linkage between such surveys and administrative data has been permitted in order to increase the overall analytic potential (Canadian Institute of Health Research, 2002). Linked person-based data from Nova Scotia and Manitoba, in conjunction with multilevel modelling, have helped address questions relevant for such population health research (Roos, Magoon, Gupta, Chateau, & Veugelers, 2003). One Manitoba research program is combining anonymized individual information from the Ministries of Family Services and Education, a measure of fetal health from maternal serum screening, and the core health data repository. Although the different ministries collected their information completely independently, rates of successful record linkage of over 90% have been achieved. The work is examining the relationship between health and educational achievement, individual

2237

characteristics, family circumstances, community contexts, school environments and the use of health-care services. Analyses will describe regions where child health outcomes and educational achievements are higher or lower than expected given an area’s socioeconomic profile. Factors that attenuate or protect against inequalities in health and educational achievement across socioeconomic levels during two key periods of childhood—early in elementary school and late in high school—will also be explored.

Discussion Successful efforts Manitoba efforts at communication to intended consumers of research have been in line with suggestions from students of the policy process (Innvaer, Vist, Trommald, & Oxman, 2002; Lavis et al., 2002). In Manitoba, interaction with the Ministry of Health and Regional Health Authorities has gone beyond the negotiation of deliverables to discussion of substance in working groups and other forums. Feedback occurs from informal briefings and during the 60-day period between presentation of a report to Manitoba Health and public release. Thus, preliminary work is brought to the attention of government officials through a variety of channels. Easy to read four-page summaries of these deliverables are provided in paper copies and on the Internet. Statistical tables in the form of EXCEL spreadsheets are on the Internet. Federally funded work at the Manitoba Centre has been directed not only toward new knowledge creation, but also toward facilitation of collaborative research with Regional Health Authorities and toward communication and dissemination activities (Martens et al., 2002). Besides the successful efforts noted earlier, the Manitoba Centre for Health Policy has influenced provincial policies in other ways. Analyses of ongoing health reform focused bed closures on the most expensive part of the health-care system: teaching and other acute care hospitals. One evaluation of rural health care showed Manitobans underused local hospitals, while many rural hospitals were keeping too long the patients they did get (Black & Burchill, 1999). Shortly after the study’s release, a pilot project with the Southeast Manitoba Rural Health Authority to serve more surgery patients at two local hospitals was announced. On the other hand, analysis led to a decrease in size (by an entire floor) of a rural hospital that had been set for expansion. Centre work has noted where real health needs exist. For example, a 1997 report on specialists pointed out that Manitobans had one of the lowest rates of hip and knee replacement in the country (Roos, Fransoo, Bogdanovic, Friesen & MacWilliam, 1999b). Within

ARTICLE IN PRESS 2238

L.L. Roos et al. / Social Science & Medicine 58 (2004) 2231–2241

two months, as a trade-off for closing maternity services at one hospital, the budget was transferred to orthopedic surgery. More recently, Manitoba was reported to have one of the highest knee replacement rates in Canada (Manitoba Health, 2002). A Centre report on the health of Manitoba’s children highlighted the startling number of childhood injuries and deaths were due to accidents (Brownell, Friesen, & Mayer, 2002). Within weeks, Manitoba Health announced a new initiative aimed at preventing childhood injuries in the home. Less successful efforts The Manitoba Centre been less successful in influencing particular practice patterns through guidelines and feedback. One deliverable, carried out in conjunction with the College of Physicians and Surgeons of Manitoba, examined rates of tonsillectomy, making comparisons across population groups and providers (hospitals and physicians)(Black, Peterson, Mansfield, & Thliveris, 1999). The number of tonsillectomies in both rural Manitoba and Winnipeg declined in 1995, the year that clinical practice guidelines and the Centre report were issued. However, by 1998 rates for rural children had increased to pre-guideline levels, with the rates for Winnipeg children remaining lower (Brownell, 2002). Unfortunately, no ongoing relationship with the College or systematic effort feeding back hospital-specific rates to surgeons has been funded. Several suggested changes in the internal workings of hospitals have not come about. Thus, despite complaints about marked delays in some elective or non-urgent surgery, surgical beds have continued to be closed during holiday periods (December, summer) (Menec et al., 2002; Rabson, 2003). Existing capacity should allow treatment of more acute care patients, despite concerns about hospital bed supply in Winnipeg. Although some improvements between 1993 and 1998 occurred, better utilization management and increased efficiencies in discharge management appear possible (Bruce, DeCoster, Trumble Waddell, & Burchill, 2002). Finally, Manitoba Health and the Winnipeg Regional Health Authority have only recently expressed a commitment to systematically examine patient safety and hospital quality of care. A Centre deliverable on this topic has been commissioned. Early work highlighted marked variation in readmission rates among hospitals with little subsequent effect (Harrison, Graff, Roos, & Brownell, 1995). Given public concerns with health care, the potential for using administrative data to improve accountability and transparency is clear (Romano et al., 2003). Generalizability The information-rich environments in parts of Australia and the United Kingdom can clearly support use

of databases for policy analysis. One recent Australian review has stressed the need to develop health information and analytical capacity in parallel to support evidence-based policy-making (van Gool, Lancsar, Viney, Hall, & Haywood, 2002). Jurisdictions without universal health insurance will find the approaches outlined here relevant when data for a defined portion of the population are available. American Medicare files have supported considerable small area research and longitudinal analyses for the population 65 and over (Fisher, Wennberg, Stukel, & Sharp, 1994; Wennberg, Fisher, & Skinner, 2002). ‘‘Enriched’’ administrative data from the American Medicare programme—the Minimum Data Set (MDS) administered in nursing homes and the Outcome and Assessment Information Set (OASIS) collected during home health-care visits—expand the research possibilities (Iezzoni, 2002). Although certain American states have made efforts to decrease the proportion of uninsured, the extent to which accurate, populationbased numerators can be constructed for the under 65 population is unclear (Kronick & Gilmer, 1999). Gaps in Canada’s data collection present problems with regard to both the specific information lacking and the attendant difficulty in understanding health care as a whole when pieces are missing. Manitoba, for example, abandoned the regular collection of person-based data from hospital emergency rooms a number of years ago because of the expense and perceived lack of utility. Recently, data gathering efforts have been initiated after publicity, in Manitoba and elsewhere, about waiting times in emergency rooms and the overflow of patients into the halls (Schull, Morrison, Vermeulan, & Redelmeier, 2003). Integrated data collection needs to accompany any innovations such as ‘‘capitation’’, often suggested as part of primary health-care reform. This is not currently a significant problem in Manitoba. Hospital data are not affected, while just 7% of physicians submit ‘‘evaluation claims’’ (claims for which remuneration is not attached) for at least a portion of their visits (Watson et al., 2003). Checks on data quality have been encouraging; some ‘‘experiments’’ on capitation involve a fee-for-service component to provide an incentive for submission of claims (Roos et al., 1996). Planning to maintain data quality is essential; in both the United Kingdom and the United States one innovation (managed competition) has been shown to result in a degraded capability for evaluation and increasingly fragmented information (Light, 1997; Welch & Welch, 1995). Manitoba’s recent investments in improving information typically address problems of significant scope: hospital overcrowding due to influenza/pneumonia and ‘‘hallway medicine’’ because of emergency room overflow. In the short term, the development of costing algorithms and the application of record linkage to existing data sets is probably even more cost effective in generating useful research.

ARTICLE IN PRESS L.L. Roos et al. / Social Science & Medicine 58 (2004) 2231–2241

Although routinely collected administrative data can provide much of what is necessary for policy analysis, there are limitations. The routine transfer of data into a research environment (such as the annual movement of files from Manitoba Health to the Manitoba Centre for Health Policy) will periodically cause timeliness to be an issue. Moreover, administrative data cannot measure patient satisfaction or the kind of ‘‘system malfunction’’ which may lead to inconvenience or poor outcomes among relatively small numbers of patients (Cleary, 2003; Rabson, 2003). Complexities in data preparation and analysis need to be systematically addressed in information-rich environments. Over the last several years, the Manitoba Centre has built an Internet-based knowledge repository to document both general concepts and software for the analysis of administrative data (see http://www.umanitoba.ca/centres/mchp/concept/). The reuse of concepts and algorithms from one study to another is facilitated by such efforts to systematize working knowledge. Recently over 10,200 hosts outside the Manitoba Centre accessed the Concept Dictionary in 1 month, even though the Manitoba data themselves can only be used through permission from university and provincial committees (Roos et al., 2003).

Conclusion The policy research (and associated provincial funding) described here has extended for over 13 years at this writing. In Manitoba, the governing party has changed while several Ministers and Deputy Ministers of Health have come and gone. A succession plan for the Manitoba Centre for Health Policy is in place for after the current leadership retires. The challenges for new leadership in Manitoba (as at other Canadian centres) will include: building on established relationships while delivering on agreed-upon analyses for the provincial Ministry of Health, obtaining access to additional data sets by the ‘‘smiling persistence’’ which has been successful in the past, and leveraging provincial funding by obtaining significant federal support. The productivity of research using government-funded data sets should help ensure support for the Canadian policy centres working with these resources (Macintyre, Chalmers, Horton, & Smith, 2001; Roos & Shapiro, 1999). The information-rich environments described here should continue to facilitate planning and managing health care for an entire population.

Acknowledgements The authors would like to thank Phyllis Jivan, JoAnne Baribeau and Carole Ouelette for their help. The

2239

assistance of the Oxford Record Linkage Study, the Western Australia Centre for Health Services Research, the Centre for Health Services and Policy Research in British Columbia, and the Institute for Clinical and Evaluative Sciences in Ontario is gratefully acknowledged. This work was supported by the Canadian Population Health Initiative and by project-related funding to the Manitoba Centre for Health Policy from Manitoba Health. Verena Menec holds a New Investigator award from the Canadian Institutes for Health Research. The results and conclusions are those of the authors and no official endorsement by Manitoba Health was intended or should be implied.

References Abbass, A. (2002). Intensive short-term dynamic psychotherapy in a private psychiatric office: Clinical and cost effectiveness. American Journal of Psychotherapy, 56(2), 225–232. Anderson, G. F., Hurst, J., Hussey, P. S., & Jee-Hughes, M. (2000). Health spending and outcomes: Trends in OECD countries, 1960–1998. Health Affairs (Millwood), 19(3), 150–157. Barer, M. L., & Stoddart, G. L. (1992). Toward integrated medical resource policies for Canada: 8. Geographic distribution of physicians. Canadian Medical Association Journal, 147(5), 617–623. Black, C., & Burchill, C. (1999). An assessment of the potential for repatriating care from urban to rural Manitoba. Medical Care, 37(Suppl. 6), JS167–JS186. Black, C., Peterson, S., Mansfield, J., & Thliveris, M. (1999). Using population-based data to enhance clinical practice guideline development. Medical Care, 37(Suppl. 6), JS254–JS263. Brownell, M. (2002). Tonsillectomy rates for Manitoba children: Temporal and spatial variations. Healthcare Management Forum, 15, Winter (Suppl), 21–26. Brownell, M., Friesen, D., & Mayer, T. (2002). Childhood injury rates in Manitoba: Sociodemographic influences. Canadian Journal of Public Health, 93(Suppl. 2), S50–S56. Brownell, M., Roos, N. P., & Roos, L. L. (2001). Monitoring health reform: A report card approach. Social Science and Medicine, 52(5), 657–670. Bruce, S., DeCoster, C., Trumble Waddell, J., & Burchill, C. (2002). Patients hospitalized for medical conditions in Winnipeg, Canada: Appropriateness and level of care. Healthcare Management Forum, 15, Winter (Suppl), 53–57. Canada Senate. (2002). The health of Canadians—The federal role. Interim Report. Volume Five: Principles and Recommendations for Reform—Part 1 [M.J.L. Kirby, Chair]. Standing Senate Committee on Social Affairs, Science and Technology, Ottawa. Canadian Institute of Health Research. (2002). Secondary use of personal information in health research: Case studies (November ed.). Ottawa, ON: Public Works and Government Services Canada. Casebeer, A., & Johnson, D. (2000). Potholes in the information highway: The use of health service utilization data by

ARTICLE IN PRESS 2240

L.L. Roos et al. / Social Science & Medicine 58 (2004) 2231–2241

Alberta Health Care managers. Healthcare Management Forum, 13(2), 58–64. Cleary, P. D. (2003). A hospitalization from hell: A patient’s perspective on quality. Annals of Internal Medicine, 138(1), 33–39. Fisher, E. S., Wennberg, J. E., Stukel, T. A., & Sharp, S. M. (1994). Hospital readmission rates for cohorts of Medicare beneficiaries in Boston and New Haven. New England Journal of Medicine, 331(15), 989–995. Forget, E. L., Deber, R. B., & Roos, L. L. (2002). Medical savings accounts: Will they reduce costs? Canadian Medical Association Journal, 167(2), 143–147. Goldacre, M. J., Griffith, M., Gill, L. E., & Mackintosh, A. (2002). In-hospital deaths as fraction of all deaths within 30 days of hospital admission for surgery: Analysis of routine statistics. British Medical Journal, 324(7345), 1069–1070. Gray, C. (2000). Hospital crisis? What crisis? Canadian Medical Association Journal, 162(7), 1043. Harrison, M. L., Graff, L. A., Roos, N. P., & Brownell, M. (1995). Discharging patients earlier from Manitoba’s hospitals: Does it adversely affect quality of care? Canadian Medical Association Journal, 153(6), 745–751. Holman, C. D. J., Bass, A. J., Rouse, I. L., & Hobbs, M. S. T. (1999). Population-based linkage of health records in Western Australia: Development of the health services research linked database. Australian New Zealand Journal of Public Health, 23(5), 453–459. Iezzoni, L. I. (2002). Using administrative data to study persons with disabilities. Milbank Quarterly, 80(2), 347–379. Iglehart, J. K. (2000). Interview: Restoring the status of an icon: A talk with Canada’s Minister of Health. Health Affairs (Millwood), 19(3), 132–140. Innvaer, S., Vist, G., Trommald, M., & Oxman, A. (2002). Health policy-makers’ perceptions of their use of evidence: A systematic review. Journal of Health Services Research Policy, 7(4), 239–244. Jacobs, P., Assiff, L., Bachynsky, J., Balada, J.-F., Botz, C. K., Brown, M., Cohen, M., Faienza, B., Finlayson, G., Hall, E., Hollander, M., Lemay, A., MacLean, N., Newson, B., O’Reilly, D., Otten, N., Robinson, C., Steeves, A., Sus, R., & Wall, M. (2000). A National List of Provincial Costs for Health Care: Canada 1997/98. Winnipeg, MB, Manitoba Centre for Health Policy. http://www.ihe.ca/download/ national costlist.pdf Jacobs, P., & Roos, N. P. (1999). Standard cost lists for health care in Canada: Issues in validity and inter provincial consolidation. Pharmacoeconomics, 15(6), 551–560. Kendrick, S. W., Douglas, M. M., Gardner, D., & Hucker, D. (1998). Best-link matching of Scottish health data sets. Methods of Information in Medicine, 37(1), 64–68. Kindig, D., & Stoddart, G. (2003). What is population health? American Journal of Public Health, 93(3), 380–383. Kobliner, B. (2003). A new health plan works, at least for the healthy. New York Times, March 2, 2003; Section 3 (p. 8) (col. 2). Kronick, R., & Gilmer, T. (1999). Explaining the decline in health insurance coverage, 1979–1995. Health Affairs (Millwood), 18(2), 30–47.

Kryger, M. H., Walld, R., & Manfreda, J. (2002). Diagnoses received by narcolepsy patients in the year prior to diagnosis by a sleep specialist. Sleep, 25(1), 36–41. Lavis, J. N., Ross, S. E., Hurley, J. E., Hohenadel, J. M., Stoddart, G. L., Woodward, C. A., & Abelson, J. (2002). Examining the role of health services research in public policymaking. Milbank Quarterly, 80(1), 125–154. Light, D. W. (1997). From managed competition to managed cooperation: Theory and lessons from the British experience. Milbank Quarterly, 75(3), 297–341. Link, B. G., Northridge, M. E., Phelan, J. C., & Ganz, M. L. (1998). Social epidemiology and the fundamental cause concept: On the structuring of effective cancer screens by socioeconomic status. Milbank Quarterly, 76(3), 375–402. Liu, L., Hader, J., Brossart, B., White, R., & Lewis, S. (2001). Impact of rural hospital closures in Saskatchewan, Canada. Social Science and Medicine, 52(12), 1793–1804. Lix, L., Newburn-Cook, C., Roos, N. P., & Derksen, S. (2002). Trends in health and health care utilization in Manitoba. Healthcare Management Forum, 15, Winter (Suppl), 35–38. Macintyre, S., Chalmers, I., Horton, R., & Smith, R. (2001). Using evidence to inform health policy: Case study. British Medical Journal, 322(7280), 222–225. Manitoba Health. (2002). Manitoba’s health indicators report. Manitoba Health, Winnipeg, MB. Martens, P., Bond, R., Jebamani, L., Burchill, C., Roos, N.P., Derksen, S., Beaulieu, M., Steinbach, C., MacWilliam, L., Walld, R., Dik, N., Sanderson, D., and the Health Information and Research Committee, Assembly of Manitoba Chiefs, Tanner-Spence, M., Leader, A., Elias, B., & O’Neil, J. (2002). The health and health care use of registered first nations people living in Manitoba: A population-based study. Winnipeg, MB: Manitoba Centre for Health Policy. Martens, P., Stewart, D., Mitchell, L., & Black, C. (2002). Assessing the performance of rural hospitals. Healthcare Management Forum, 15, Winter (Suppl), 27–34. Melton, L. J. I. (1996). History of the Rochester epidemiology project. Mayo Clinic Proceedings, 71(3), 266–274. Menec, V., Black, C., Roos, N. P., & Bogdanovic, B. (2001). What is the potential for formal patient registration in Canadian primary care? The scale of ‘informal registration’ in Manitoba. Journal of Health Services Research Policy, 6(4), 202–206. Menec, V. H., Roos, N. P., & MacWilliam, L. (2002). Seasonal patterns of hospital use in Winnipeg: Implications for managing winter bed crises. Healthcare Management Forum, 15, Winter (Suppl), 58–64. Mustard, C. A., Derksen, S., Berthelot, J.-M., & Wolfson, M. C. (1999). Assessing ecologic proxies for household income: A comparison of household and neighbourhood-level income measures in the study of population health status. Health and Place, 5(2), 157–171. Mustard, C. A., Derksen, S., & Black, C. (1999). Widening regional inequality in premature mortality rates in Manitoba. Canadian Journal of Public Health, 90(6), 372–376. Naylor, C. D., & Slaughter, P. M. (1999). Cardiovascular health & services in Ontario. An ICES atlas. Toronto, ON: Institute for Clinical Evaluative Sciences. Rabson, M. (2003). Heart-surgery wait claims 3 lives. (Vol. A5). Winnipeg: Free Press.

ARTICLE IN PRESS L.L. Roos et al. / Social Science & Medicine 58 (2004) 2231–2241 Rankin, J. M., Spinelli, J. J., Carere, R. G., Ricci, D. R., Penn, I. M., Hilton, J. D., Henderson, M. A., Hayden, R. I., & Buller, C. E. (1999). Improved clinical outcome after widespread use of coronary-artery stenting in Canada. New England Journal of Medicine, 341(26), 1957–1965. Reid, R. A., Bogdanovic, B., Roos, N. P., Black, C., MacWilliam, L., & Menec, V. (2001). Do some physician groups see sicker patients than others? Implications for primary care policy in Manitoba. Winnipeg, MB: Manitoba Centre for Health Policy and Evaluation. Roberts, J. D., Fransoo, R., Black, C., Roos, L. L., & Martens, P. (2002). Research meets reality: Administrative data to guide planning for Canadian Regional Health Authorities. Healthcare Management Forum, 15(4), 13–21. Robinson, J. R., Young, T. K., Roos, L. L., & Gelskey, D. E. (1997). Estimating the burden of disease: Comparing administrative data and self-reports. Medical Care, 35(9), 932–947. Romano, P. S., Geppert, J. J., Davies, S., Miller, M. R., Elixhauser, A., & McDonald, K. M. (2003). A national profile of patient safety in US hospitals. Health Affairs (Millwood), 22(2), 154–166. Romanow, R. J. (2002). Commission on the future of health care in Canada. Ottawa, ON: Government of Canada. Roos, L. L., Magoon, J., Gupta, S., Chateau, D., & Veugelers, P. J. (2003). Socioeconomic determinants of mortality in two Canadian provinces: Multilevel modelling and neighbourhood context. Presented at the Canadian Population Society 2003 annual meeting, socio-economic status and health session, Dalhousie University, Halifax NS, June 2003. Roos, L. L., & Nicol, J. P. (1999). A research registry: Uses, development, and accuracy. Journal of Clinical Epidemiology, 52(1), 39–47. Roos, L. L., Soodeen, R., Bond, R., & Burchill, C. (2003). Working more productively: Tools for administrative data. Health Services Research, in press. Roos, N. P., Fransoo, R., Bogdanovic, B., Carriere, K. C., Frohlich, N., Friesen, D., Patton, D., & Wall, R. (1999a). Needs-based planning for generalist physicians. Medical Care, 37(Suppl. 6), JS206–JS228. Roos, N. P., Fransoo, R., Bogdanovic, B., Friesen, D., Frohlich, N., Carriere, K. C., Patton, D., & Wall, R. (1996). Needs-based planning for Manitoba’s generalist physicians. Winnipeg, MB: Manitoba Centre for Health Policy and Evaluation. Roos, N. P., Fransoo, R., Bogdanovic, B., Friesen, D., & MacWilliam, L. (1999b). Issues in planning for specialist physicians. Medical Care, 37(Suppl. 6), JS229–JS253. Roos, N. P., & Shapiro, E. (1999). Academics at the policy interface: Revisiting the Manitoba centre for health policy

2241

and evaluation and its population-based health information system. Medical Care, 37(Suppl. 6), JS1–JS308. Rosman, D. L. (2001). The Western Australian road injury database (1987–1996): Ten years of linked police, hospital and death records of road crashes and injuries. Accident; Analysis Prevention, 33(1), 81–88. Saltman, R. B., & Figueras, J. (1998). Analyzing the evidence on European health care reforms. Health Affairs (Millwood), 17(2), 85–108. Schull, M. J., Morrison, L. J., Vermeulan, M., & Redelmeier, D. A. (2003). Emergency department overcrowding and ambulance transport delays for patients with chest pain. Canadian Medical Association Journal, 168(3), 277–283. Shanahan, M., Loyd, M., Roos, N. P., & Brownell, M. (1999). A comparative study of the costliness of Manitoba hospitals. Medical Care, 37(Suppl. 6), JS101–JS122. Shanahan, M., Steinbach, C., Burchill, C., Friesen, D., & Black, C. (1999). Adding up provincial expenditures on health care for Manitobans. Medical Care, 37(Suppl. 6), JS60–JS82. Sheps, S. B., Reid, R. J., Barer, M. L., Krueger, H., McGrail, K. M., Green, B., Evans, R. G., & Hertzman, C. (2000). Hospital downsizing and trends in health care use among elderly people in British Columbia. Canadian Medical Association Journal, 163(4), 411–412. Soodeen, R., Roos, L. L., & Peterson, S. (2000). Health reform and technological change: Shifting hospitalization patterns for four procedures in Manitoba. Healthcare Management Forum, 13(1), 15–21. van Gool, K., Lancsar, E., Viney, R., Hall, J., & Haywood, P. (2002). Diagnosis and prognosis of Australia’s health information for evidence-based policy. Journal of Health Services Research Policy, 7(Suppl. 1), S1:40–S1:45. Watson, D., Bogdanovic, B., Heppner, P., Katz, A., Reid, R., & Roos, N. P. (2004). The use of physician services by older adults: Temporal trends 1991/92 to 2000/01. Canadian Journal of Aging, in press. Welch, W. P., & Welch, H. G. (1995). Medicare analysis—Feefor-data: A strategy to open the HMO black box. Health Affairs (Millwood), 14(4), 104–116. Wennberg, J. E., & Cooper, M. M. (1999). The quality of medical care in the United States: A report on the medicare program, The Dartmouth atlas of health care. Chicago, IL: American Health Association Press. Wennberg, J. E., Fisher, E. S., & Skinner, J. S. (2002). Geography and the debate over medicare reform. Health Affairs (Millwood), 21, (February 13, 2002), W96–W114. Wharry, S., & Sibbald, B. (2002). What physician shortage? Canadian Medical Association Journal, 167(1), 64.