- Email: [email protected]
Questions remain over validity of EUROCARE data
World Report
Questions remain over validity of EUROCARE data With a general election looming in the UK, some experts are concerned about the use of cancer survival comparisons in the debate over the country’s cancer services. Emma Wilkinson reports.
Science Photo Library
For the results of EUROCARE-4 see Lancet Oncol 2007; 8: 773–83 and Lancet Oncol 2007; 8: 784–96
It does not take a general election for the UK’s National Health Service (NHS) to be used as a political football, but debate about funding and quality of service is likely to be a prominent one in the coming months. Health care in the UK is an emotive and powerful topic at home and abroad, as proven by the recent reaction to discussions on the NHS in the US political debate on health-care reform. Cancer care in particular seems to grab the headlines, and comparisons with other countries in Europe are often used to highlight poor quality services. Yet some experts are raising grave concerns about how cancer survival data from studies such as the European Cancer Registry-based study 4 (EUROCARE-4), published in 2007, could be misrepresented to paint the NHS in a poorer light than is in fact the reality. Fears that such data could be used unfairly by politicians has prompted Cancer Research UK—the country’s leading investor in cancer research and information campaigns—to review its position on EUROCARE. In a meeting held at the beginning of this month, the organisation
The introduction of screening programmes can affect cancer survival comparisons
964
concluded it wanted to be clearer about the dangers of straight comparisons or league tables as a way in which to criticise the UK’s record on cancer care. Sarah Woolnaugh, head of policy at Cancer Research UK, said they were conscious in the lead up to an election that cancer outcomes were going to be a key issue. “We wanted to get a
“...comparisons can only really be made between the UK and other countries with full national cancer registration data, such as Sweden, Finland, and Norway.” sense of how we might best be talking about EUROCARE data”, she says. “In terms of survival data, EUROCARE is the best we have but there have been concerns about how the data are sometimes presented.” She said that the main conclusion of the meeting, which was attended by experts in cancer epidemiology among others, was that comparisons can only really be made between the UK and other countries with full national cancer registration data, such as Sweden, Finland, and Norway. On the face of it, the UK has poorer outcomes than say France or Germany, but cancer registration data only cover 17% and 1%, respectively, of those countries, making comparisons difficult, she said. “What we are slightly concerned about is where people have tried to present the data as a league table when there are these important caveats to consider—we have to consider the data very carefully.” She also stressed that they would be briefing members of parliament on the fact that EUROCARE data cannot yet show whether the Cancer Plan, introduced in 2000, has had an effect. “We don’t want the politicians to look
at this and say we’re not doing well”, she adds. The charity is now planning to issue new guidelines on the use of cancer survival comparisons on their website but say that they are looking forward to seeing the EUROCARE-5 data, due some time in 2011. The accuracy of cancer outcome data is a surprisingly emotive topic, and one that leads to strong differences of opinion. Robert Souhami, emeritus professor of medicine at University College London and former director of clinical research at Cancer Research UK, feels very emphatically that much of the data within EUROCARE are unreliable. “It exercises me greatly that the data have been used to criticise cancer care in the UK—it has even been used to suggest that the health service should be privatised.” His main concern is that the data used are not representative of the whole population in many of the countries assessed. “How can you determine the survival rate in Germany if you only know about 1% of cases, we have no idea if that is representative of the country as a whole”, he says. The fact that UK cases make up 40% of the EUROCARE-4 dataset should “set alarm bells ringing”, he adds. Other problems that he believes add to the unreliability of the data are lack of information about ethnic origin, which can affect the prognosis of some cancers, and issues raised by the introduction of screening programmes. He says the public are those most affected when the data are used to draw conclusions that simply cannot be reached. “It’s very important the British press stop beating up cancer services in the UK on the basis of this study that is not convincingly accurate. It’s really upsetting when people see this and www.thelancet.com Vol 374 September 19, 2009
ask if they should be going abroad for treatment, it’s not a trivial issue.” For Souhami to be convinced that the UK does indeed lag behind its European neighbours on cancer survival he would want to see convincing data for screen-detected cancers, ethnic origin, and separate analyses discounting those countries where data coverage was poor. Julian Peto, professor of cancer epidemiology at the London School of Hygiene and Tropical Medicine, agrees that such comparison studies are fraught with difficulties. “The basic problem is that there are all sorts of biases in cancer registration data. The potential for bias is enormous— there is the issue of early versus late diagnosis and the effect of screening, and a whole string of problems on whether the data are representative— and it’s very difficult to know how to deal with these issues. For example, the reduction in mortality from breast cancer is a real effect but that is a combination of treatment and screening and it’s quite hard to pick it to pieces”, he says. “It’s not something from which we can draw firm conclusions and it justifies looking into more closely.” The Department of Health agrees and is about to launch several studies looking into reasons behind apparent differences in survival, concentrating on countries where they believe the data are comparable, such as the Nordic countries, Canada, and Australia. Mike Richards, national clinical director for cancer, attended the Cancer Research UK meeting to discuss EUROCARE, but he says that despite some of the issues with the data, he is happy that the differences reported are true. “I had no problem with the EUROCARE-4 paper and I took it very seriously. I think in broad terms there is a consensus that the differences between countries are to a large extent real differences and much of that can be attributed to late diagnosis.” Where he does take issue is use of the data to condemn the effectiveness www.thelancet.com Vol 374 September 19, 2009
of the national Cancer Plan, which was launched the year after the study period in question in EUROCARE-4. He says that comparing the UK with Scandinavian countries, where cancer registration is just as comprehensive, does show that the UK is lagging, at least it was up until 2000. “1-year survival rates are poor, which is also a reflection of late diagnosis”, he adds. “We now need to work with a small number of other countries as well as doing EUROCARE-5 to explore these differences in great depth and that’s something the Department of Health is sponsoring.”
“I think...that the differences between countries are to a large extent real differences and much of that can be attributed to late diagnosis.” “We will be getting more up-to-date comparisons and looking at differences in public awareness, primary care, and access to diagnostics.” In EUROCARE-4 it is fair to say that the gap between survival rates in the UK and other western European countries seemed to be narrowing, and with recent investment it would be hoped that further improvement would be apparent in the next EUROCARE round. But Richards does not believe the gap has yet been eliminated. “It is important to remember that it takes time to turn big tankers round.” Franco Berrino from the department of preventive and predictive medicine at the Fondazione Istituto Nazionale dei Tumori, Italy, and EUROCARE study leader, is well aware of the debate around the validity of survival comparisons that has been particularly heated in the UK. But he says that usually the conclusion has been that the observed differences are largely true. Moreover, the reliability of the results should not be the main topic for debate but the underlying reasons for the differences. “Instead of losing time
David Sandison/The Independent
World Report
Mike Richards, UK clinical director for cancer, backs the findings of EUROCARE-4
discrediting the reliability of the observed survival difference, it would be better to invest in better studies to clarify why they do exist.” The deadline for EUROCARE-5 data collection is next summer, with the results expected the following year. But, says Gordon Wishart, consultant breast and endocrine surgeon in Cambridge, there has been much improvement in the past 5–10 years. More up-to-date figures from the east of England on breast cancer patients diagnosed between 2000 and 2002 showed that for women younger than 65 years survival is as good if not better than other countries in the EUROCARE dataset. They did identify poorer outcomes after the age of 65 years and are now working to address why that might be. “I’m always wary of league tables and I do get very frustrated with these comparisons and that’s one of the reason I’m trying to get to the bottom of what’s going on. We found that overall we were very good and I think investment in recent years has allowed us to catch up with Europe.” Cancer Research UK, at least, hopes that it will be able to convince the politicians to look beyond the league tables, consider a range of measures of success, and work out what is really needed to build on the investment in NHS cancer services.
Emma Wilkinson 965
Questions remain over validity of EUROCARE data With a general election looming in the UK, some experts are concerned about the use of cancer survival comparisons in the debate over the country’s cancer services. Emma Wilkinson reports.
Science Photo Library
For the results of EUROCARE-4 see Lancet Oncol 2007; 8: 773–83 and Lancet Oncol 2007; 8: 784–96
It does not take a general election for the UK’s National Health Service (NHS) to be used as a political football, but debate about funding and quality of service is likely to be a prominent one in the coming months. Health care in the UK is an emotive and powerful topic at home and abroad, as proven by the recent reaction to discussions on the NHS in the US political debate on health-care reform. Cancer care in particular seems to grab the headlines, and comparisons with other countries in Europe are often used to highlight poor quality services. Yet some experts are raising grave concerns about how cancer survival data from studies such as the European Cancer Registry-based study 4 (EUROCARE-4), published in 2007, could be misrepresented to paint the NHS in a poorer light than is in fact the reality. Fears that such data could be used unfairly by politicians has prompted Cancer Research UK—the country’s leading investor in cancer research and information campaigns—to review its position on EUROCARE. In a meeting held at the beginning of this month, the organisation
The introduction of screening programmes can affect cancer survival comparisons
964
concluded it wanted to be clearer about the dangers of straight comparisons or league tables as a way in which to criticise the UK’s record on cancer care. Sarah Woolnaugh, head of policy at Cancer Research UK, said they were conscious in the lead up to an election that cancer outcomes were going to be a key issue. “We wanted to get a
“...comparisons can only really be made between the UK and other countries with full national cancer registration data, such as Sweden, Finland, and Norway.” sense of how we might best be talking about EUROCARE data”, she says. “In terms of survival data, EUROCARE is the best we have but there have been concerns about how the data are sometimes presented.” She said that the main conclusion of the meeting, which was attended by experts in cancer epidemiology among others, was that comparisons can only really be made between the UK and other countries with full national cancer registration data, such as Sweden, Finland, and Norway. On the face of it, the UK has poorer outcomes than say France or Germany, but cancer registration data only cover 17% and 1%, respectively, of those countries, making comparisons difficult, she said. “What we are slightly concerned about is where people have tried to present the data as a league table when there are these important caveats to consider—we have to consider the data very carefully.” She also stressed that they would be briefing members of parliament on the fact that EUROCARE data cannot yet show whether the Cancer Plan, introduced in 2000, has had an effect. “We don’t want the politicians to look
at this and say we’re not doing well”, she adds. The charity is now planning to issue new guidelines on the use of cancer survival comparisons on their website but say that they are looking forward to seeing the EUROCARE-5 data, due some time in 2011. The accuracy of cancer outcome data is a surprisingly emotive topic, and one that leads to strong differences of opinion. Robert Souhami, emeritus professor of medicine at University College London and former director of clinical research at Cancer Research UK, feels very emphatically that much of the data within EUROCARE are unreliable. “It exercises me greatly that the data have been used to criticise cancer care in the UK—it has even been used to suggest that the health service should be privatised.” His main concern is that the data used are not representative of the whole population in many of the countries assessed. “How can you determine the survival rate in Germany if you only know about 1% of cases, we have no idea if that is representative of the country as a whole”, he says. The fact that UK cases make up 40% of the EUROCARE-4 dataset should “set alarm bells ringing”, he adds. Other problems that he believes add to the unreliability of the data are lack of information about ethnic origin, which can affect the prognosis of some cancers, and issues raised by the introduction of screening programmes. He says the public are those most affected when the data are used to draw conclusions that simply cannot be reached. “It’s very important the British press stop beating up cancer services in the UK on the basis of this study that is not convincingly accurate. It’s really upsetting when people see this and www.thelancet.com Vol 374 September 19, 2009
ask if they should be going abroad for treatment, it’s not a trivial issue.” For Souhami to be convinced that the UK does indeed lag behind its European neighbours on cancer survival he would want to see convincing data for screen-detected cancers, ethnic origin, and separate analyses discounting those countries where data coverage was poor. Julian Peto, professor of cancer epidemiology at the London School of Hygiene and Tropical Medicine, agrees that such comparison studies are fraught with difficulties. “The basic problem is that there are all sorts of biases in cancer registration data. The potential for bias is enormous— there is the issue of early versus late diagnosis and the effect of screening, and a whole string of problems on whether the data are representative— and it’s very difficult to know how to deal with these issues. For example, the reduction in mortality from breast cancer is a real effect but that is a combination of treatment and screening and it’s quite hard to pick it to pieces”, he says. “It’s not something from which we can draw firm conclusions and it justifies looking into more closely.” The Department of Health agrees and is about to launch several studies looking into reasons behind apparent differences in survival, concentrating on countries where they believe the data are comparable, such as the Nordic countries, Canada, and Australia. Mike Richards, national clinical director for cancer, attended the Cancer Research UK meeting to discuss EUROCARE, but he says that despite some of the issues with the data, he is happy that the differences reported are true. “I had no problem with the EUROCARE-4 paper and I took it very seriously. I think in broad terms there is a consensus that the differences between countries are to a large extent real differences and much of that can be attributed to late diagnosis.” Where he does take issue is use of the data to condemn the effectiveness www.thelancet.com Vol 374 September 19, 2009
of the national Cancer Plan, which was launched the year after the study period in question in EUROCARE-4. He says that comparing the UK with Scandinavian countries, where cancer registration is just as comprehensive, does show that the UK is lagging, at least it was up until 2000. “1-year survival rates are poor, which is also a reflection of late diagnosis”, he adds. “We now need to work with a small number of other countries as well as doing EUROCARE-5 to explore these differences in great depth and that’s something the Department of Health is sponsoring.”
“I think...that the differences between countries are to a large extent real differences and much of that can be attributed to late diagnosis.” “We will be getting more up-to-date comparisons and looking at differences in public awareness, primary care, and access to diagnostics.” In EUROCARE-4 it is fair to say that the gap between survival rates in the UK and other western European countries seemed to be narrowing, and with recent investment it would be hoped that further improvement would be apparent in the next EUROCARE round. But Richards does not believe the gap has yet been eliminated. “It is important to remember that it takes time to turn big tankers round.” Franco Berrino from the department of preventive and predictive medicine at the Fondazione Istituto Nazionale dei Tumori, Italy, and EUROCARE study leader, is well aware of the debate around the validity of survival comparisons that has been particularly heated in the UK. But he says that usually the conclusion has been that the observed differences are largely true. Moreover, the reliability of the results should not be the main topic for debate but the underlying reasons for the differences. “Instead of losing time
David Sandison/The Independent
World Report
Mike Richards, UK clinical director for cancer, backs the findings of EUROCARE-4
discrediting the reliability of the observed survival difference, it would be better to invest in better studies to clarify why they do exist.” The deadline for EUROCARE-5 data collection is next summer, with the results expected the following year. But, says Gordon Wishart, consultant breast and endocrine surgeon in Cambridge, there has been much improvement in the past 5–10 years. More up-to-date figures from the east of England on breast cancer patients diagnosed between 2000 and 2002 showed that for women younger than 65 years survival is as good if not better than other countries in the EUROCARE dataset. They did identify poorer outcomes after the age of 65 years and are now working to address why that might be. “I’m always wary of league tables and I do get very frustrated with these comparisons and that’s one of the reason I’m trying to get to the bottom of what’s going on. We found that overall we were very good and I think investment in recent years has allowed us to catch up with Europe.” Cancer Research UK, at least, hopes that it will be able to convince the politicians to look beyond the league tables, consider a range of measures of success, and work out what is really needed to build on the investment in NHS cancer services.
Emma Wilkinson 965