- Email: [email protected]
English health profiles – did they do what was expected? An evaluation of Health Profiles 2006
Public Health 123 (2009) 311–315
Contents lists available at ScienceDirect
Public Health journal homepage: www.elsevierhealth.com/journals/pubh
Original Research
English health profiles – did they do what was expected? An evaluation of Health Profiles 2006 C. Bradford a, A. Hill b, J. Wilkinson a, * a b
North East Public Health Observatory, University of Durham Queen’s Campus, Wolfson Research Institute, Stockton on Tees TS17 6BH, UK South East Public Health Observatory, Oxford, UK
a r t i c l e i n f o
s u m m a r y
Article history: Received 14 January 2008 Received in revised form 22 December 2008 Accepted 28 January 2009 Available online 19 March 2009
Objectives: To assess the impact of Health Profiles 2006 by English local authorities, and to determine what changes need to be made to the profiles to have an impact on their target audience.
Keywords: Health profiles Evaluation Health improvement Local authorities
Study design: A telephone- and web-based survey of a sample of the health profiles’ target audience was conducted, along with an analysis of web statistics. Methods: In total, 285 telephone interviews were undertaken. Fifty-three percent of the respondents were National Health Service employees. An evaluation form was also available through the Health Profiles website, which elicited 117 responses (19 scrutiny officers and 83 members of the public). Results: There was a positive response to the content and format of Health Profiles 2006. The majority of respondents felt that the profiles provided a unique summary of local-authority-based health and health inequality information that was both accessible and understandable. Conclusions: A number of recommendations are made to improve health profile production as a tool for information for health improvement. These include simplifying graphics, greater local input into commentary, and a more interactive website so that the data can be accessed and explored. However, the recommendations for change must be viewed alongside the number of comments specifically relating to the English health profiles that nothing should be changed other than updating the data and filling in the gaps. Ó 2009 The Royal Society for Public Health. Published by Elsevier Ltd. All rights reserved.
Background Creating public health reports has been one of the mainstays of public health for centuries. The Bills of Mortality1 in the 17th Century are some of the oldest. Medical officers of health within local authorities had a duty to report on the health of their population. Until the abolition of the role of medical officer of health, these reports were a regular feature of local authority activity. They did become somewhat routine, but owing to the independence of medical officers of health (they could only be removed by the sanction of the House of Lords), they were often hard-hitting. More recently, directors of public health have been required to produce an annual report on the health of their population.2 These, while very welcome, have tended to be very diverse in format and content. As such, it has been difficult for users to compare areas, with no standardization of reports and no attempt to standardize the data contained in these reports. In 2003, the Faculty of Public
* Corresponding author. Tel.: þ44 191 334 0400; fax: þ44 191 334 0391. E-mail address: [email protected] (J. Wilkinson).
Health attempted to develop good practice in this area by the publication of guidance on the production and content of annual reports by directors of public health.3 It was because of this diversity that, in 2004, Choosing Health4 contained a commitment to publish a standard set of localauthority-level reports. In 2005, the Association of Public Health Observatories was commissioned by the Department of Health in England to produce the first series of health profiles. These were published in 2006.5 Key drivers for the production of health profiles include the recognition that information must be easily accessible and the messages must be easily understood. Increased partnership working between primary care trusts, local authorities, police services and third sector organizations has highlighted the need for clear, concise health information that can be understood by all partners. There is also increasing recognition that there is a need for greater understanding of health inequalities both within geographies, e.g. local authorities, and between geographies. Until the production of a core national set of health profiles, this has required individual partnerships to develop and analyse their own core datasets to assess local inequalities. Benchmarking across geographies has not been undertaken systematically.
0033-3506/$ – see front matter Ó 2009 The Royal Society for Public Health. Published by Elsevier Ltd. All rights reserved. doi:10.1016/j.puhe.2009.01.010
312
C. Bradford et al. / Public Health 123 (2009) 311–315
There is a lack of peer-reviewed literature on the production of effective health profiles in order to compare subnational geographies. The English Health Profile Project therefore was based on informal information from other health profiles which had been produced (Evaluation of the Scottish health profiles – unpublished). The evaluation therefore is a mechanism to improve production of the English health profiles, and add to the absent literature on effective profiles. The European Union (EU) ISARE6 Project has sought to collate health indicator data at subnational level for all countries in the EU. The successor project I2SARE (commenced in 2008) is to turn these subnational health indicators into EU health profiles. These EU health profiles are seen as vital in order to inform the identification and analysis of health inequalities across the EU and the distribution of EU structural funds. The experience of the English health profiles can contribute to this process. The experience of the English health profiles has already been used to inform health profile development in Scotland, Portugal and the Netherlands. It was always recognized that there would be a long development process before the ‘perfect health profile’ was published, and indeed the first set of profiles was described as a ‘prototype’. As part of this development, there was an agreement to undertake an evaluation of the first health profiles in order to inform the development of profiles in subsequent years. This paper contains the result of this evaluation and recommendations for effective health profile development.
dissemination of the profiles, with an aim of at least 50% being local authority members and staff. This was because local authority members and their staff were the key audience for the profiles. PHOs collated the responses to their regional surveys, and these were subsequently collated centrally by the Health Profiles Project team. Website survey The questions from the telephone survey were added to the website (www.communityhealthprofiles.info) for online completion. Users could also provide feedback via the ‘contact us’ link on the website if they did not want to complete the online survey form. Results PHOs did not meet the target for respondents to the telephone survey. A total of 285 telephone interviews were carried out, representing 74% of the original target, with 53% of respondents being from the NHS. The difficulties encountered by the PHOs included timing (summer holidays), lack of willingness of local authority staff to take part when contacted by PHOs, and lack of awareness of the health profiles among those contacted. There were 117 responses from the website. Identification of common themes
Introduction Health Profiles 2006 (prototype) were released to the National Health Service (NHS) and local government in June 2006. No formal media launch was undertaken. This represented the first publication to produce standardized health information for 386 of the 388 local authorities in England. The City of London and Isles of Scilly were not covered due to data limitations as a result of their small population size (note the ‘City of London’ – otherwise known as the ‘square mile’ – is the business heart of the capital and has a resident population of less than 10 000). In the initial commission, it was agreed that the health profiles would be updated annually. In order to improve content and usability of the health profiles for subsequent years, a qualitative evaluation of users and stakeholders was undertaken. The purpose of this evaluation was to provide timely evidence-based and informed information to improve production of the health profiles in the following years, and to contribute to the literature on the production of effective health profiles. Methods In this evaluation, two approaches were taken: public health observatory (PHO) interviews with users and stakeholders, and collation of responses to the survey via a website. The evaluation was undertaken between August and October 2006. PHO interviews All the PHOs in England conducted semi-structured interviews with stakeholders and users using a standardized form that had been designed by the health profiles team (available from the website). The interview structure was based on an outcomes evaluation method based on the original outcomes specified in the Health Profiles Project plan. It was piloted in two regions before being rolled out across all nine PHOs in the region. The target number of interviews was 385 (one respondent for each health profile produced). PHOs were asked to identify users to interview, by random selection from those on the distribution list used locally for
The health profiles were generally found to be useful documents and were considered to be helpful for local prioritization work and partnership working. In particular, they were described as being highly valued by those working in local authorities who often struggle to get access to data that are seen as being ‘owned’ by the NHS: ‘I find it very difficult to access relevant information and whenever I ask anyone in the NHS for it, they tell me that drawing down such information from the myriad systems used by the NHS is not easy or even always possible.’ (Health scrutiny officer) ‘Before the recently published health profiles, it was very difficult to source high-quality and up-to-date information.’ (Scrutiny support officer) There was poor penetration and awareness of the health profiles in local authorities and other sectors. In some regions, it was clear during the survey process to identify participants that the ‘mail out’ system alone had been unsuccessful in reaching the key target audience, leading to inability and reluctance of local authority partners to participate. There were many comments that the profiles needed better publicity, both in advance of their publication and afterwards. However, where a direct alert system had been used (via the local authority scrutiny officer network), good awareness was achieved. The poor penetration and awareness of the profiles may have led to difficulty in pursuing local-authority-based evaluation in a number of regions. The message from the majority of respondents was that the health profiles did supply a concise and accessible summary of local health, and facilitated comparison within and between geographies. The profiles were considered to be easy to use, a good idea, informative, concise and accessible: ‘I liked the fact that they were readable in 15 minutes or so and gave the headline figures.’ (Scrutiny officer) The following were the most commonly stated positive aspects of the profiles: clear messages, easy to understand; maps;
C. Bradford et al. / Public Health 123 (2009) 311–315
national comparisons; a health summary; length and format; and the presence of ‘key points’ – a good balance between positive and negative messages.
However, the less popular aspects of the profiles were: a confusing health summary; lack of timeliness of (some of) the data; the choice of certain indicators; maps and concern about the use of ‘low tide’ boundaries and some of the ward boundaries; use of ‘best’ and ‘worst’ and the absence in some instances of local issues; visually difficult to use (font, shading on maps); and difficulty in accessing further information from the website.
Website evaluation responses There were 117 respondents via the website (88 members of the public and 29 local authority staff). In the subsequent analysis, all totals do not add up to 117 as not all respondents answered all questions. Public responses Forty respondents stated that they already had easy access to information on health and health inequalities, and 40 respondents stated that they did not. Forty-one respondents found the health profiles ‘very easy’ or ‘easy’ to understand, 28 found the profiles ‘neither easy nor difficult’ to understand, and five found the health profiles ‘difficult’ to understand. Nine respondents found the health profiles ‘very difficult’ to understand, and all specified that this was because the website was not working at the time. Nine respondents stated that they were unable to access the information that they wanted due to the technical failure of the website: ‘The website is very slow and unresponsive – and doesn’t work, obviously overloaded. If you are going to do a publicity launch via the media, at least make sure your website can handle the interest/ traffic!’ ‘I’m disappointed in this service, it’s completely unusable, poor management.’ Two respondents made comments regarding the lack of information for Wales and Scotland, although the profiles were never designed to cover these countries. A comment was also received about the missing information for indicators on the health profiles. These indicators were specifically included to highlight the lack of data for these important health indicators to both inform the audience that the data was (often surprisingly) not available at local-authority level, and also to act as a lobbying tool to increase the collation of important data to inform action on health inequalities. It is noted that these indicators have largely been made available for Health Profiles 2008: ‘It’s a shame there were some missing statistics in various areas as I would have found the statistics on items such as breast feeding or drinking during pregnancy very interesting.’ However, a number of positive comments were made: ‘Good to see the data. A few more comparative graphs could be interesting but this is a very helpful start.’
313
‘Clear understanding of what is happening in the community.’ ‘Just fascinating.’ ‘Gives a rounded picture of principal health indicators for particular areas against regional and national figures.’ ‘I found it to be very informative and liked the fact that it informed me about my local area and that I was able to look at other areas.’ ‘More detail is always appreciated – search by town or postcode.’ ‘Back-up analysis (more detail) would be useful, but would naturally tend to provide information overload and detract from the provision of a succinct and easily digestible format. For example: road deaths in St Albans district are higher than average – why? Is it because of the M1 motorway? Or particularly high density of traffic?’ ‘Need to continue to develop and populate current blank areas, and data to be as current as possible.’ ‘I found the health profiles to be quite informative as they demonstrated the inequalities by ward, which I think is key in truly understanding one’s area.’ ‘Very informative. Would like to be able to download data into a spreadsheet or .csv file.’ ‘I particularly like the fact that they are available and well set out. Make it easier/clearer for web user to find them.’ ‘Short even combination of maps and charts and text. Good quality notes.’ ‘The fact that they are an up-to-date publication.’
Professional responses Of the local authority responses (n ¼ 29), 19 of the scrutiny officers reported that they had easy access to health information, as did five other respondents. Nine scrutiny officers found the profiles ‘very easy’ or ‘easy’ to interpret, as did all of the other respondents. None of the professional respondents found the health profiles ‘difficult’ or ‘very difficult’ to understand: ‘People without access to the Internet and there are still many particularly in C... have no idea that such information is available. This exercise should be started in schools and print-outs taken home for the parents to read and understand.’ ‘Clear, easy to understand – particularly summary. Looking forward to when data are available for the GAP indicators.’
Application and utilization of the health profiles Questions were asked about how the health profiles had been used, and numerous examples were given of cases where the profiles had already been used within the NHS, local authorities and elsewhere. These include: annual reports by directors of public health; health partnership work including identification of public health priorities and strategy setting; local area agreements; performance and target monitoring in both NHS and local authorities; scrutiny committee work; local strategic partnerships; practice-based commissioning; support for funding bids: National Lottery Wellbeing bid; big lottery bid; needs assessment; sustainable communities events; and voluntary sector: Age Concern meeting.
314
C. Bradford et al. / Public Health 123 (2009) 311–315
‘The profiles helped to clarify what are the key issues in terms of health priorities. The Council has for the first time got ‘‘improving health of the local population’’ as one of its key (13) objectives’ ‘This is all we need, the health of our local population is so bad it will just stigmatize us even more’ (Local councillor) ‘It gave a useful snapshot of the key health issues in the area and current trends. It showed where further action is needed to improve performance against national trends.’ (Scrutiny policy officer) ‘Wards with levels of high deprivation have been highlighted for councillors to determine specific health needs so that these can be focused on in health scrutiny.’ (Scrutiny policy officer)
For effective dissemination of health profiles, authors should ensure that there is a clear agreed communications strategy to ensure that: the target audience is well aware of the release of the new profiles and the updated website; the target audience is informed of the potential use of the profiles and the data on the website and is supported in its use; PHOs and other intelligence organizations and other partners are aware of the media launch and can maximize the health improvement impact with local campaigns; and the website is able to cope with increased traffic associated with a successful launch.
Recommendations for effective health profile production The clear message from the evaluation was that in order to be effective, there must be a clear understanding of who is the intended target audience. In the case of the English health profiles, this evaluation has found that the general format and content were right for the intended target audience (local authority members and officers), including the interested public with Internet access (who were not the target audience). The majority of respondents found the health profiles ‘very easy’ or ‘easy’ to understand. Therefore, the recommendations for effective health profile production and improvements for Health Profiles 2007 need to be seen as refinements rather than wholesale revision of the ‘prototype’ format and content. Health profiles enable ready access to summary population health and wellbeing information at local-authority level. The key messages and information in the profiles were readily available and accessible to those (including the interested public) who have not had ready access to the information in a useable format. However, in order to be effective, a comprehensive communication strategy is required. Those with public health and NHS expertise found the profiles less useful for their own work, as they frequently had ready access to the data as well as more local information. However, the public health community found the profiles to be useful in partnership working and production of reports, including annual reports by directors of public health.
Presentation of health profiles The comments received on the overall presentation of the profiles included suggestions to: maintain the concise nature of the paper document; simplify the health summary chart; provide more data at sub-local-authority level (preferably at super-output-area level); improve the quality of the maps; provide trend data in future profiles; and use the usual red, green and amber traffic lights (rather than just red and amber). In terms of modifying the indicators themselves, suggestions were made to: fill in gaps (especially childhood indicators); add in all age cause mortality; add in more information on health inequalities; clarify the indicators on mental health and air quality; explain the rationale for including the indicator on educational achievement in a set of health indicators; amend the older persons indicator; and add a health protection indicator.
Communications strategy During the development of the profiles, a number of changes were made in the communications strategy. As a result, the profiles were released to the NHS and local authorities, and on the Internet (www.communityhealthprofiles.info) in June 2006 with no media launch. This significantly impaired general awareness of the presence and use of health profiles. In October 2006, after completion of the data collection for this evaluation, ‘Health Challenge England’7 and the publication of the first health profile for England, the ‘National Health Profile’8 resulted in high levels of national and local media interest in the local health profiles, with a dramatic increase in the numbers accessing the community health profiles website and downloading health profiles. Within the first week after ‘Health Challenge England’, there had been more than 40,000 referrals from the BBC website and more than 240,000 downloads of health profiles. This is in comparison to an average 2000 hits per day in the weeks before ‘Health Challenge England’. A number of adverse comments were received from the public due to the technical problems. The responses about the website from the public highlight that both public and professional users need to be considered in a future communications strategy, even if the public is not the intended primary audience.
Interpretation and commentary The following comments were received on the interpretation and the commentary: there is a need to improve the commentary to help with the interpretation of maps and charts; advice on how to reduce inequalities should be ‘signposted’; and there should be more local input and information in the commentary, including more positive information.
Availability of data Whilst maintaining the concise nature of the summary document, make further information available for those who are interested or need it from an easy-to-access web format. Allow for comparisons to be made between areas (possibly from a website). Improve explanatory information/metadata.
C. Bradford et al. / Public Health 123 (2009) 311–315
315
Discussion
Acknowledgements
This qualitative evaluation of Health Profiles 2006 provided a valuable insight into the target audience’s views of the new Health Profiles Project. Overall, the comments were positive and, where less so, contained constructive comments for the future development of the profiles. The fact that the evaluation was led by the PHOs (the producers) could have led to bias in the comments received about the profiles or from the selection of participants. However, there was no evidence to suggest that this had happened, and indeed the PHOs had actively sought less than positive comments as these often contain the most useful pointers for improvement. An external evaluation was considered and rejected as it was felt likely to add little additional information following the release of the ‘prototype’ profiles. However, an external evaluation is currently being commissioned (2008). There are some clear overall messages about the nature of the profiles (short, succinct and well designed), together with the process (early consultation with a clear target group is essential). Some of the areas of concern were outside the control of the PHOs – in particular, the unplanned exposure of the website details which occurred as a result of a BBC television programme related to a new government initiative on health improvement. This simply emphasized the need for all relevant infrastructure and to ensure that websites are able to cope with extreme demand. For the future, there is already a commitment that health profiles will be published annually. There is a need to develop a more flexible interactive website. However, given the positive feedback, any change needs to be undertaken carefully. Health profiles have been a large success in England. There are ways in which these could be improved, but care needs to be taken to avoid making too many changes which will affect ‘brand recognition’.
The authors are grateful to all the staff in the PHOs who contributed to the development of these health profiles and for their help in the evaluation. The authors also wish to thank all those who responded to the consultation and made the evaluation possible. Ethical approval None sought. Funding The Health Profiles Project was funded by the Department of Health. Competing interests Claire Bradford, Alison Hill and John Wilkinson are all employed by PHOs in England which receive their core funding from the Department of Health. References 1. Coates E. A collection of all Bills of Mortality. London; 1665. 2. Gorsky M. Local leadership in public health. J Epidemiol Community Health 2007;61:468–72. 3. Hill A. Guidance on the production and content of annual reports for directors of public health in primary care trusts. London: Faculty of Public Health, Public Health Resource Unit and Association of Public Health Observatories; 2003. 4. Department of Health. Choosing health – making healthier choices easier. Department of Health. Cm 6374. Available at: http://www.dh.gov.uk/en/ Publicationsandstatistics/Publications/PublicationsPolicyAndGuidance/DH_ 4094550; 2004 [accessed 19.12.08]. 5. Association of Public Health Observatories. Health profiles for England. York: Association of Public Health Observatories; 2007. 6. Wilkinson JR, Berghmans L, Imbert F, Lede´sert B, Ochoa A. Health indicators in the European regions – ISAREII. Eur J Public Health 2008;18:178–83. 7. Department of Health. Health Challenge England. London: Department of Health; 2006. 8. Department of Health. Health profile of England. London: Department of Health; 2007.
Contents lists available at ScienceDirect
Public Health journal homepage: www.elsevierhealth.com/journals/pubh
Original Research
English health profiles – did they do what was expected? An evaluation of Health Profiles 2006 C. Bradford a, A. Hill b, J. Wilkinson a, * a b
North East Public Health Observatory, University of Durham Queen’s Campus, Wolfson Research Institute, Stockton on Tees TS17 6BH, UK South East Public Health Observatory, Oxford, UK
a r t i c l e i n f o
s u m m a r y
Article history: Received 14 January 2008 Received in revised form 22 December 2008 Accepted 28 January 2009 Available online 19 March 2009
Objectives: To assess the impact of Health Profiles 2006 by English local authorities, and to determine what changes need to be made to the profiles to have an impact on their target audience.
Keywords: Health profiles Evaluation Health improvement Local authorities
Study design: A telephone- and web-based survey of a sample of the health profiles’ target audience was conducted, along with an analysis of web statistics. Methods: In total, 285 telephone interviews were undertaken. Fifty-three percent of the respondents were National Health Service employees. An evaluation form was also available through the Health Profiles website, which elicited 117 responses (19 scrutiny officers and 83 members of the public). Results: There was a positive response to the content and format of Health Profiles 2006. The majority of respondents felt that the profiles provided a unique summary of local-authority-based health and health inequality information that was both accessible and understandable. Conclusions: A number of recommendations are made to improve health profile production as a tool for information for health improvement. These include simplifying graphics, greater local input into commentary, and a more interactive website so that the data can be accessed and explored. However, the recommendations for change must be viewed alongside the number of comments specifically relating to the English health profiles that nothing should be changed other than updating the data and filling in the gaps. Ó 2009 The Royal Society for Public Health. Published by Elsevier Ltd. All rights reserved.
Background Creating public health reports has been one of the mainstays of public health for centuries. The Bills of Mortality1 in the 17th Century are some of the oldest. Medical officers of health within local authorities had a duty to report on the health of their population. Until the abolition of the role of medical officer of health, these reports were a regular feature of local authority activity. They did become somewhat routine, but owing to the independence of medical officers of health (they could only be removed by the sanction of the House of Lords), they were often hard-hitting. More recently, directors of public health have been required to produce an annual report on the health of their population.2 These, while very welcome, have tended to be very diverse in format and content. As such, it has been difficult for users to compare areas, with no standardization of reports and no attempt to standardize the data contained in these reports. In 2003, the Faculty of Public
* Corresponding author. Tel.: þ44 191 334 0400; fax: þ44 191 334 0391. E-mail address: [email protected] (J. Wilkinson).
Health attempted to develop good practice in this area by the publication of guidance on the production and content of annual reports by directors of public health.3 It was because of this diversity that, in 2004, Choosing Health4 contained a commitment to publish a standard set of localauthority-level reports. In 2005, the Association of Public Health Observatories was commissioned by the Department of Health in England to produce the first series of health profiles. These were published in 2006.5 Key drivers for the production of health profiles include the recognition that information must be easily accessible and the messages must be easily understood. Increased partnership working between primary care trusts, local authorities, police services and third sector organizations has highlighted the need for clear, concise health information that can be understood by all partners. There is also increasing recognition that there is a need for greater understanding of health inequalities both within geographies, e.g. local authorities, and between geographies. Until the production of a core national set of health profiles, this has required individual partnerships to develop and analyse their own core datasets to assess local inequalities. Benchmarking across geographies has not been undertaken systematically.
0033-3506/$ – see front matter Ó 2009 The Royal Society for Public Health. Published by Elsevier Ltd. All rights reserved. doi:10.1016/j.puhe.2009.01.010
312
C. Bradford et al. / Public Health 123 (2009) 311–315
There is a lack of peer-reviewed literature on the production of effective health profiles in order to compare subnational geographies. The English Health Profile Project therefore was based on informal information from other health profiles which had been produced (Evaluation of the Scottish health profiles – unpublished). The evaluation therefore is a mechanism to improve production of the English health profiles, and add to the absent literature on effective profiles. The European Union (EU) ISARE6 Project has sought to collate health indicator data at subnational level for all countries in the EU. The successor project I2SARE (commenced in 2008) is to turn these subnational health indicators into EU health profiles. These EU health profiles are seen as vital in order to inform the identification and analysis of health inequalities across the EU and the distribution of EU structural funds. The experience of the English health profiles can contribute to this process. The experience of the English health profiles has already been used to inform health profile development in Scotland, Portugal and the Netherlands. It was always recognized that there would be a long development process before the ‘perfect health profile’ was published, and indeed the first set of profiles was described as a ‘prototype’. As part of this development, there was an agreement to undertake an evaluation of the first health profiles in order to inform the development of profiles in subsequent years. This paper contains the result of this evaluation and recommendations for effective health profile development.
dissemination of the profiles, with an aim of at least 50% being local authority members and staff. This was because local authority members and their staff were the key audience for the profiles. PHOs collated the responses to their regional surveys, and these were subsequently collated centrally by the Health Profiles Project team. Website survey The questions from the telephone survey were added to the website (www.communityhealthprofiles.info) for online completion. Users could also provide feedback via the ‘contact us’ link on the website if they did not want to complete the online survey form. Results PHOs did not meet the target for respondents to the telephone survey. A total of 285 telephone interviews were carried out, representing 74% of the original target, with 53% of respondents being from the NHS. The difficulties encountered by the PHOs included timing (summer holidays), lack of willingness of local authority staff to take part when contacted by PHOs, and lack of awareness of the health profiles among those contacted. There were 117 responses from the website. Identification of common themes
Introduction Health Profiles 2006 (prototype) were released to the National Health Service (NHS) and local government in June 2006. No formal media launch was undertaken. This represented the first publication to produce standardized health information for 386 of the 388 local authorities in England. The City of London and Isles of Scilly were not covered due to data limitations as a result of their small population size (note the ‘City of London’ – otherwise known as the ‘square mile’ – is the business heart of the capital and has a resident population of less than 10 000). In the initial commission, it was agreed that the health profiles would be updated annually. In order to improve content and usability of the health profiles for subsequent years, a qualitative evaluation of users and stakeholders was undertaken. The purpose of this evaluation was to provide timely evidence-based and informed information to improve production of the health profiles in the following years, and to contribute to the literature on the production of effective health profiles. Methods In this evaluation, two approaches were taken: public health observatory (PHO) interviews with users and stakeholders, and collation of responses to the survey via a website. The evaluation was undertaken between August and October 2006. PHO interviews All the PHOs in England conducted semi-structured interviews with stakeholders and users using a standardized form that had been designed by the health profiles team (available from the website). The interview structure was based on an outcomes evaluation method based on the original outcomes specified in the Health Profiles Project plan. It was piloted in two regions before being rolled out across all nine PHOs in the region. The target number of interviews was 385 (one respondent for each health profile produced). PHOs were asked to identify users to interview, by random selection from those on the distribution list used locally for
The health profiles were generally found to be useful documents and were considered to be helpful for local prioritization work and partnership working. In particular, they were described as being highly valued by those working in local authorities who often struggle to get access to data that are seen as being ‘owned’ by the NHS: ‘I find it very difficult to access relevant information and whenever I ask anyone in the NHS for it, they tell me that drawing down such information from the myriad systems used by the NHS is not easy or even always possible.’ (Health scrutiny officer) ‘Before the recently published health profiles, it was very difficult to source high-quality and up-to-date information.’ (Scrutiny support officer) There was poor penetration and awareness of the health profiles in local authorities and other sectors. In some regions, it was clear during the survey process to identify participants that the ‘mail out’ system alone had been unsuccessful in reaching the key target audience, leading to inability and reluctance of local authority partners to participate. There were many comments that the profiles needed better publicity, both in advance of their publication and afterwards. However, where a direct alert system had been used (via the local authority scrutiny officer network), good awareness was achieved. The poor penetration and awareness of the profiles may have led to difficulty in pursuing local-authority-based evaluation in a number of regions. The message from the majority of respondents was that the health profiles did supply a concise and accessible summary of local health, and facilitated comparison within and between geographies. The profiles were considered to be easy to use, a good idea, informative, concise and accessible: ‘I liked the fact that they were readable in 15 minutes or so and gave the headline figures.’ (Scrutiny officer) The following were the most commonly stated positive aspects of the profiles: clear messages, easy to understand; maps;
C. Bradford et al. / Public Health 123 (2009) 311–315
national comparisons; a health summary; length and format; and the presence of ‘key points’ – a good balance between positive and negative messages.
However, the less popular aspects of the profiles were: a confusing health summary; lack of timeliness of (some of) the data; the choice of certain indicators; maps and concern about the use of ‘low tide’ boundaries and some of the ward boundaries; use of ‘best’ and ‘worst’ and the absence in some instances of local issues; visually difficult to use (font, shading on maps); and difficulty in accessing further information from the website.
Website evaluation responses There were 117 respondents via the website (88 members of the public and 29 local authority staff). In the subsequent analysis, all totals do not add up to 117 as not all respondents answered all questions. Public responses Forty respondents stated that they already had easy access to information on health and health inequalities, and 40 respondents stated that they did not. Forty-one respondents found the health profiles ‘very easy’ or ‘easy’ to understand, 28 found the profiles ‘neither easy nor difficult’ to understand, and five found the health profiles ‘difficult’ to understand. Nine respondents found the health profiles ‘very difficult’ to understand, and all specified that this was because the website was not working at the time. Nine respondents stated that they were unable to access the information that they wanted due to the technical failure of the website: ‘The website is very slow and unresponsive – and doesn’t work, obviously overloaded. If you are going to do a publicity launch via the media, at least make sure your website can handle the interest/ traffic!’ ‘I’m disappointed in this service, it’s completely unusable, poor management.’ Two respondents made comments regarding the lack of information for Wales and Scotland, although the profiles were never designed to cover these countries. A comment was also received about the missing information for indicators on the health profiles. These indicators were specifically included to highlight the lack of data for these important health indicators to both inform the audience that the data was (often surprisingly) not available at local-authority level, and also to act as a lobbying tool to increase the collation of important data to inform action on health inequalities. It is noted that these indicators have largely been made available for Health Profiles 2008: ‘It’s a shame there were some missing statistics in various areas as I would have found the statistics on items such as breast feeding or drinking during pregnancy very interesting.’ However, a number of positive comments were made: ‘Good to see the data. A few more comparative graphs could be interesting but this is a very helpful start.’
313
‘Clear understanding of what is happening in the community.’ ‘Just fascinating.’ ‘Gives a rounded picture of principal health indicators for particular areas against regional and national figures.’ ‘I found it to be very informative and liked the fact that it informed me about my local area and that I was able to look at other areas.’ ‘More detail is always appreciated – search by town or postcode.’ ‘Back-up analysis (more detail) would be useful, but would naturally tend to provide information overload and detract from the provision of a succinct and easily digestible format. For example: road deaths in St Albans district are higher than average – why? Is it because of the M1 motorway? Or particularly high density of traffic?’ ‘Need to continue to develop and populate current blank areas, and data to be as current as possible.’ ‘I found the health profiles to be quite informative as they demonstrated the inequalities by ward, which I think is key in truly understanding one’s area.’ ‘Very informative. Would like to be able to download data into a spreadsheet or .csv file.’ ‘I particularly like the fact that they are available and well set out. Make it easier/clearer for web user to find them.’ ‘Short even combination of maps and charts and text. Good quality notes.’ ‘The fact that they are an up-to-date publication.’
Professional responses Of the local authority responses (n ¼ 29), 19 of the scrutiny officers reported that they had easy access to health information, as did five other respondents. Nine scrutiny officers found the profiles ‘very easy’ or ‘easy’ to interpret, as did all of the other respondents. None of the professional respondents found the health profiles ‘difficult’ or ‘very difficult’ to understand: ‘People without access to the Internet and there are still many particularly in C... have no idea that such information is available. This exercise should be started in schools and print-outs taken home for the parents to read and understand.’ ‘Clear, easy to understand – particularly summary. Looking forward to when data are available for the GAP indicators.’
Application and utilization of the health profiles Questions were asked about how the health profiles had been used, and numerous examples were given of cases where the profiles had already been used within the NHS, local authorities and elsewhere. These include: annual reports by directors of public health; health partnership work including identification of public health priorities and strategy setting; local area agreements; performance and target monitoring in both NHS and local authorities; scrutiny committee work; local strategic partnerships; practice-based commissioning; support for funding bids: National Lottery Wellbeing bid; big lottery bid; needs assessment; sustainable communities events; and voluntary sector: Age Concern meeting.
314
C. Bradford et al. / Public Health 123 (2009) 311–315
‘The profiles helped to clarify what are the key issues in terms of health priorities. The Council has for the first time got ‘‘improving health of the local population’’ as one of its key (13) objectives’ ‘This is all we need, the health of our local population is so bad it will just stigmatize us even more’ (Local councillor) ‘It gave a useful snapshot of the key health issues in the area and current trends. It showed where further action is needed to improve performance against national trends.’ (Scrutiny policy officer) ‘Wards with levels of high deprivation have been highlighted for councillors to determine specific health needs so that these can be focused on in health scrutiny.’ (Scrutiny policy officer)
For effective dissemination of health profiles, authors should ensure that there is a clear agreed communications strategy to ensure that: the target audience is well aware of the release of the new profiles and the updated website; the target audience is informed of the potential use of the profiles and the data on the website and is supported in its use; PHOs and other intelligence organizations and other partners are aware of the media launch and can maximize the health improvement impact with local campaigns; and the website is able to cope with increased traffic associated with a successful launch.
Recommendations for effective health profile production The clear message from the evaluation was that in order to be effective, there must be a clear understanding of who is the intended target audience. In the case of the English health profiles, this evaluation has found that the general format and content were right for the intended target audience (local authority members and officers), including the interested public with Internet access (who were not the target audience). The majority of respondents found the health profiles ‘very easy’ or ‘easy’ to understand. Therefore, the recommendations for effective health profile production and improvements for Health Profiles 2007 need to be seen as refinements rather than wholesale revision of the ‘prototype’ format and content. Health profiles enable ready access to summary population health and wellbeing information at local-authority level. The key messages and information in the profiles were readily available and accessible to those (including the interested public) who have not had ready access to the information in a useable format. However, in order to be effective, a comprehensive communication strategy is required. Those with public health and NHS expertise found the profiles less useful for their own work, as they frequently had ready access to the data as well as more local information. However, the public health community found the profiles to be useful in partnership working and production of reports, including annual reports by directors of public health.
Presentation of health profiles The comments received on the overall presentation of the profiles included suggestions to: maintain the concise nature of the paper document; simplify the health summary chart; provide more data at sub-local-authority level (preferably at super-output-area level); improve the quality of the maps; provide trend data in future profiles; and use the usual red, green and amber traffic lights (rather than just red and amber). In terms of modifying the indicators themselves, suggestions were made to: fill in gaps (especially childhood indicators); add in all age cause mortality; add in more information on health inequalities; clarify the indicators on mental health and air quality; explain the rationale for including the indicator on educational achievement in a set of health indicators; amend the older persons indicator; and add a health protection indicator.
Communications strategy During the development of the profiles, a number of changes were made in the communications strategy. As a result, the profiles were released to the NHS and local authorities, and on the Internet (www.communityhealthprofiles.info) in June 2006 with no media launch. This significantly impaired general awareness of the presence and use of health profiles. In October 2006, after completion of the data collection for this evaluation, ‘Health Challenge England’7 and the publication of the first health profile for England, the ‘National Health Profile’8 resulted in high levels of national and local media interest in the local health profiles, with a dramatic increase in the numbers accessing the community health profiles website and downloading health profiles. Within the first week after ‘Health Challenge England’, there had been more than 40,000 referrals from the BBC website and more than 240,000 downloads of health profiles. This is in comparison to an average 2000 hits per day in the weeks before ‘Health Challenge England’. A number of adverse comments were received from the public due to the technical problems. The responses about the website from the public highlight that both public and professional users need to be considered in a future communications strategy, even if the public is not the intended primary audience.
Interpretation and commentary The following comments were received on the interpretation and the commentary: there is a need to improve the commentary to help with the interpretation of maps and charts; advice on how to reduce inequalities should be ‘signposted’; and there should be more local input and information in the commentary, including more positive information.
Availability of data Whilst maintaining the concise nature of the summary document, make further information available for those who are interested or need it from an easy-to-access web format. Allow for comparisons to be made between areas (possibly from a website). Improve explanatory information/metadata.
C. Bradford et al. / Public Health 123 (2009) 311–315
315
Discussion
Acknowledgements
This qualitative evaluation of Health Profiles 2006 provided a valuable insight into the target audience’s views of the new Health Profiles Project. Overall, the comments were positive and, where less so, contained constructive comments for the future development of the profiles. The fact that the evaluation was led by the PHOs (the producers) could have led to bias in the comments received about the profiles or from the selection of participants. However, there was no evidence to suggest that this had happened, and indeed the PHOs had actively sought less than positive comments as these often contain the most useful pointers for improvement. An external evaluation was considered and rejected as it was felt likely to add little additional information following the release of the ‘prototype’ profiles. However, an external evaluation is currently being commissioned (2008). There are some clear overall messages about the nature of the profiles (short, succinct and well designed), together with the process (early consultation with a clear target group is essential). Some of the areas of concern were outside the control of the PHOs – in particular, the unplanned exposure of the website details which occurred as a result of a BBC television programme related to a new government initiative on health improvement. This simply emphasized the need for all relevant infrastructure and to ensure that websites are able to cope with extreme demand. For the future, there is already a commitment that health profiles will be published annually. There is a need to develop a more flexible interactive website. However, given the positive feedback, any change needs to be undertaken carefully. Health profiles have been a large success in England. There are ways in which these could be improved, but care needs to be taken to avoid making too many changes which will affect ‘brand recognition’.
The authors are grateful to all the staff in the PHOs who contributed to the development of these health profiles and for their help in the evaluation. The authors also wish to thank all those who responded to the consultation and made the evaluation possible. Ethical approval None sought. Funding The Health Profiles Project was funded by the Department of Health. Competing interests Claire Bradford, Alison Hill and John Wilkinson are all employed by PHOs in England which receive their core funding from the Department of Health. References 1. Coates E. A collection of all Bills of Mortality. London; 1665. 2. Gorsky M. Local leadership in public health. J Epidemiol Community Health 2007;61:468–72. 3. Hill A. Guidance on the production and content of annual reports for directors of public health in primary care trusts. London: Faculty of Public Health, Public Health Resource Unit and Association of Public Health Observatories; 2003. 4. Department of Health. Choosing health – making healthier choices easier. Department of Health. Cm 6374. Available at: http://www.dh.gov.uk/en/ Publicationsandstatistics/Publications/PublicationsPolicyAndGuidance/DH_ 4094550; 2004 [accessed 19.12.08]. 5. Association of Public Health Observatories. Health profiles for England. York: Association of Public Health Observatories; 2007. 6. Wilkinson JR, Berghmans L, Imbert F, Lede´sert B, Ochoa A. Health indicators in the European regions – ISAREII. Eur J Public Health 2008;18:178–83. 7. Department of Health. Health Challenge England. London: Department of Health; 2006. 8. Department of Health. Health profile of England. London: Department of Health; 2007.