- Email: [email protected]
Epidemiology on trial—confessions of an expert witness
COMMENTARY
1st-year malaria GFATM-funding in malaria endemic countries, south of Sahara4 Total funding (US$ millions)*
Population at risk (millions)†
Approved Benin Madagascar Mali Senegal Tanzania (mainland) Tanzania (Zanzibar) Zimbabwe Subtotal
1·483 0·639 1·454 1·429 4·157 0·409 4·746 14·317
6·27 13·88 11·20 9·42 31·45 0·97 12·12
Approved, deferred Zambia
8·400
9·9
2 3 4 5
Total funds for Africa
22·717
Not funded Eritrea Ethiopia Ghana Kenya Nigeria Malawi Mozambique Swaziland Uganda Burkina Faso Chad Democratic Rep of Congo Namibia Rwanda The Gambia
5·635 20·511 1·167 13·940 16·140 4·456 32·172 1·034 14·077 0·200 0·432 6·533 2·817 4·620 0·244
Subtotal
Funding per person at risk (US$)
1
0·24 0·05 0·13 0·15 0·13 0·42 0·39
6
7
8
3·20 41·60 19·31 14·55 113·82 10·83 18·18 0·76 20·69 11·54 7·53 46·24 1·20 3·72 1·30
Commission on Macroeconomic and Health. Improving the health outcomes of the poor: Report Of Working Group 5. Geneva: WHO, April, 2002. Roll Back Malaria: a global partnership. http://www.rbm.who.int (accessed Sept 12, 2002). WHO. The African summit on Roll Back Malaria, Abuja, April 2000. Geneva: WHO, WHO/CDS/RBM/2000.17. The Global Fund to fight AIDS, Tuberculosis and Malaria. http://www.globalfundatm.org (accessed Sept 12, 2002). Ramsay S. Global Fund makes historic first round of payments. Lancet 2002; 359: 1581–82. Craig MH, Snow RW, le Sueur D. African climatic model of malaria transmission based on monthly rainfall and temperature. Parasitol Today 1999; 15: 105–11. Deichmann U. African population database documentation. National Center for Geographic Information and Analysis, Santa Barbara, California, USA, 1996. http://grid2.cr.usgs.gov/globalpop/ africa/ (accessed Sept 12, 2002). United Nations Population Division. World population prospects: population database. http://esa.un.org/unpp (accessed Sept 12, 2002).
0·85
1·76 0·49 0·06 0·95 0·14 0·41 1·77 1·36 0·68 0·02 0·06 2·35 2·34 1·24 0·19
123·980
*For proposal not on award list, sums are requests not awards. †Populations at risk derived from climate-driven malaria-risk models,6 interpolated population distributions,7 and national 2000 population estimates.8 Populations exposed to risk include areas with acutely seasonal or epidemic risks through to intense stable transmission.
Four countries outside Africa (China, Indonesia, Laos, and Sri Lanka), which bear only 3% of the world’s malaria burden, would have access to 44% of the malaria funds awarded in this first cycle of funding. Support per person for antimalaria actions approved by GFATM remains paltry given the needs of Africa. Many national proposals for prevention and control received no support (panel). Proposals endorsed for funding by the Board will provide on average US$0·17 per person at risk during the first year. At this level, the targets agreed at Abuja have no chance of being met. The current cautionary policy of the Global Fund, which impels it to invest mainly in projects that pursue limited goals, will impede national RBM efforts. Malaria cannot afford to remain a poor cousin to the other diseases of poverty within GFATM’s funding process. The international community cannot afford to let the Fund become another missed opportunity to make a difference. *A Teklehaimanot, R W Snow *Center for Global Health and Economic Development, Columbia University, New York, NY 10002, USA; KEMRI/Wellcome Trust Collaborative Program, Nairobi, Kenya; and Centre for Tropical Medicine, University of Oxford, UK (e-mail: [email protected])
THE LANCET • Vol 360 • September 21, 2002 • www.thelancet.com
Epidemiology on trial—confessions of an expert witness A recent case in the UK High Court judged a claim by users of the third-generation oral contraceptive pill (3GOC), who alleged that the drug had caused a venous thromboembolism.1 I was an expert witness for the claimants, arguing for a scientific consensus.2 The judgment pronounced at two levels. Overall, the claimants lost. The judgment concluded that 3GOCs are not associated with a higher risk of venous thromboembolism than second-generation pills (2GOCs). This judgment rested on one argument3 and the judge therefore added a fallback position, which may be about right quantitatively: the relative risk of 3GOCs compared with 2GOCs for venous thromboembolism is around 1·7, and causal. Nevertheless, this relative risk was still not deemed sufficient to accept the claim, because all parties had pre-agreed that the relative risk had to be higher than 2·0 for it to be more likely than not that, for women who were harmed, the venous thromboembolism was due to use of a 3GOC. The evidence produced for no excess risk at all for 3GOCs was inconclusive, but it impressed the judge. A 1999 report3 purported to reduce a relative risk of 1·5 from the Transnational Study4 to 0·8 by appropriate adjustment for time-dependent variables with Cox proportional hazards modelling.5 The idea was to eliminate bias if women who are susceptible to venous thromboembolism changed to the newer oral contraceptives more readily, or succumb early in their first use of such a drug. Few understood the report at the time because detail was lacking, and relative risks rarely change direction by adjustment. After the analyses were made available, the claimants’ epidemiologist Alec Walker could explain with clarity that the variables used for adjustment were scientifically illegitimate. In the conversion from a case-control study (which the Transnational was) to a cohort study (which Cox’s regression requires), design constraints were ignored and discussion of the nature of a risk set, defined in all textbooks, was misrepresented. The adjusting variables related to the time of recruitment, and were current-use patterns of oral contraceptives, but used retrospectively. Using variables to adjust for spontaneous timedependent risk (in 1993, for example) measured in future time (1995 possibly) amounts to clairvoyance. Proper adjustment made little difference to the risk estimates, and could better have been done with 889
For personal use. Only reproduce with permission from The Lancet Publishing Group.
COMMENTARY
Meta-analyses of studies of combined oral contraceptives and venous thromboembolism* Study
Year
Adjusted 95% CI odds ratio
Relative weight
All studies Bloemenkamp WHO Transnational Farmer, Mediplus Herings, all users Parkin Jick, GPRD Lidegaard Pooled estimate
1995 1995 1996 1999 1999 2000 2000 2002
2·20 2·70 1·73 1·35 2·30 2·92 2·20 1·40 1·89
0·90–5·40 1·60–4·60 1·07–2·79 0·74–2·39 1·50–3·70 0·78–12·0 1·50–3·30 1·00–1·90 1·59–2·25
5 14 17 11 21 2 26 34 131
Industry-funded studies Transnational 1996 Farmer, Mediplus 1999 Farmer, GPRD 2000 Lidegaard 2002 Pooled estimate
1·73 1·35 1·20 1·40 1·35
1·07–2·79 0·74–2·39 0·90–1·70 1·00–1·90 1·12–1·63
17 11 46 34 108
Independent studies Bloemenkamp 1995 WHO 1995 Herings, all users 1999 Parkin 2000 Jick GPRD 2000 Pooled estimate
2·20 2·70 2·30 2·92 2·20 2·27
0·90–5·40 1·60–4·60 1·50–3·70 0·78–12·0 1·50–3·30 1·72–3·00
5 14 21 2 26 68
*Fixed-effect modelling. References for these studies can be derived from ref 8 except where actual results quoted were amended in Court1 to make them more mutually consistent. Exception is Lidegaard et al, 2002.7
nested case-control study. Jick and colleagues10 used the GPRD in the same way, but these analyses yield statistically different estimates (p<0·02). Farmer’s analysis is excluded overall, since it duplicates that of Jick. Wyeth had also undertaken an analysis of the GPRD,11 which was contested. This analysis remains unpublished and as such the judge deemed it irrelevant as evidence. So much for publication bias.12 Pharmacovigilance has to rely as much on research investment to refute an uncomfortable association, as on a search towards a reasonable scientific consensus.13 The actual biases that could explain the highly significant differences in source of funding remain unresolved, since all true details of execution are never reported. The report of Lewis and colleagues3 demonstrates that peer review guarantees little. The judge had the power to determine beyond doubt whether this analysis3 was correct. But one expert each was to be examined on each substantive question, which prevented discussion once the main expert for the claimants had returned to the USA and the defendant’s expert, asked to do more analyses, had died. The outcome is an unhappy one—appropriate safeguards for consumers of these products demand that a scientific consensual determination of risk is reached. The actual relative risk could still be over 2, since relative quality is ignored herein. The legal system in this case was unable to grapple with the scientific issues. Despite potential shortcomings in the claimants’ case, this trial and debate was a lost opportunity for pharmacoepidemiology. I have claimed fees from the claimants’ lawyers. The judge was critical of parts of my evidence. I am a member of the UK Committee on Safety of Medicines.
Klim McPherson conventional analyses of the original data, just as the WHO study did.6 Despite the fallback relative risk of 1·7 probably being about right, the judgment was wrong. The threshold of a twofold relative risk was incorrectly treated as representing the balance of probability argument. However, when comparing women on different choices of oral contraceptive, baseline risk among non-users has to be included properly. The balance of probability argument depends on absolute, not relative, risks; the risk of 2GOCs is already around three times baseline. A twofold relative risk requires the extra risk of using a 3GOC to exceed both the extra risk of 2GOC use plus the common baseline risk. This case should therefore have been won on its fallback assessment, with a greater extra absolute risk for 3GOCs than for 2GOCs, which is achieved with a relative risk larger than 1·67. In the Court, acceptable common characteristics for relevant studies changed according to an emergent consensus, and the amended results of all major studies agreed (panel). A revised report that appeared during the hearing7 is also included. The weights are calculated from the width of the confidence intervals. A woman on a 3GOC with a venous thromboembolism (relative risk 3GOC/2GOC, 1·89) has, on the balance of probability, had her venous thromboembolism more likely than not because of that oral contraceptive. The difference8 in estimates by source of funding is the major contribution to the heterogeneity between these results, here p<0·001. Two studies by Farmer and colleagues are included, one of which uses data from the General Practice Research Database (GPRD)9 for a 890
Department of Social Medicine, University of Bristol, Canynge Hall, Bristol BS8 2PR, UK (e-mail: [email protected])
1
2 3
4
5 6 7
8
9
10
11 12 13
High Courts of Justice. Case number 0002638, between XYZ & Others and Schering Health Care Limited, Organon Laboratories Limited and John Wyeth & Borther Limited. Approved Judgement, July 29, 2002. http://www.courtservice.gov.uk/judgmentsfiles/j1298/ xyz_-v-schering.htm (accessed Sept 17, 2002). Editorial. Epidemiology on trial. Lancet 2002; 360: 421. Lewis MA, MacRae KD, Kuhl-Habich DK, et al. The differential risk of OC’s; impact of full exposure history. Hum Reprod 1999; 14: 1493–199. Spitzer WO, Lewis MA, Heineman LA et al. Third generation oral contraceptives and risk of VTE: international case control study. BMJ 1996; 312: 83–87. Cox DR, Oakes D. Analysis of survival data. New York: Wiley, 1984. Poulter NR, Farley TM, Chang CL, et al. Safety of combined oral contraceptive pills. Lancet 1996; 347: 547. Lidegaard O, Edstrom B, Kreiner S. Oral contraceptives and venous thromboembolism: a five-year national case control study. Contraception 2002; 65: 187–96. Kemmeren JM, Algra A, Grobbee DE. Third generation oral contraceptives and risk of venous thrombosis: meta-analysis. BMJ 2001; 323: 131–34. Farmer RD, Williams TJ, Simpson EL, et al. Effect of 1995 pill scare on rates of venous thromboembolism among women taking combined oral contraceptives: analysis of general practice research database. BMJ 2000; 321: 477–79. Jick H, Kaye JA, Vasilakis-Scaramozza C, et al. Risk of VTE before and after 1995: cohort and case control study. BMJ 2000; 321: 1190–95. Brock P. Wyeth responds to news story on oral contraceptives and DVT. BMJ 2001; 322: 1605. Skegg D. Pitfalls of pharmacoepidemiology. BMJ 2000; 321: 1171–72. World Health Organization. Cardiovascular disease and steroid hormone contraception. Report of WHO scientific group. WHO technical reports series 877. Geneva: WHO, 1998.
THE LANCET • Vol 360 • September 21, 2002 • www.thelancet.com
For personal use. Only reproduce with permission from The Lancet Publishing Group.
1st-year malaria GFATM-funding in malaria endemic countries, south of Sahara4 Total funding (US$ millions)*
Population at risk (millions)†
Approved Benin Madagascar Mali Senegal Tanzania (mainland) Tanzania (Zanzibar) Zimbabwe Subtotal
1·483 0·639 1·454 1·429 4·157 0·409 4·746 14·317
6·27 13·88 11·20 9·42 31·45 0·97 12·12
Approved, deferred Zambia
8·400
9·9
2 3 4 5
Total funds for Africa
22·717
Not funded Eritrea Ethiopia Ghana Kenya Nigeria Malawi Mozambique Swaziland Uganda Burkina Faso Chad Democratic Rep of Congo Namibia Rwanda The Gambia
5·635 20·511 1·167 13·940 16·140 4·456 32·172 1·034 14·077 0·200 0·432 6·533 2·817 4·620 0·244
Subtotal
Funding per person at risk (US$)
1
0·24 0·05 0·13 0·15 0·13 0·42 0·39
6
7
8
3·20 41·60 19·31 14·55 113·82 10·83 18·18 0·76 20·69 11·54 7·53 46·24 1·20 3·72 1·30
Commission on Macroeconomic and Health. Improving the health outcomes of the poor: Report Of Working Group 5. Geneva: WHO, April, 2002. Roll Back Malaria: a global partnership. http://www.rbm.who.int (accessed Sept 12, 2002). WHO. The African summit on Roll Back Malaria, Abuja, April 2000. Geneva: WHO, WHO/CDS/RBM/2000.17. The Global Fund to fight AIDS, Tuberculosis and Malaria. http://www.globalfundatm.org (accessed Sept 12, 2002). Ramsay S. Global Fund makes historic first round of payments. Lancet 2002; 359: 1581–82. Craig MH, Snow RW, le Sueur D. African climatic model of malaria transmission based on monthly rainfall and temperature. Parasitol Today 1999; 15: 105–11. Deichmann U. African population database documentation. National Center for Geographic Information and Analysis, Santa Barbara, California, USA, 1996. http://grid2.cr.usgs.gov/globalpop/ africa/ (accessed Sept 12, 2002). United Nations Population Division. World population prospects: population database. http://esa.un.org/unpp (accessed Sept 12, 2002).
0·85
1·76 0·49 0·06 0·95 0·14 0·41 1·77 1·36 0·68 0·02 0·06 2·35 2·34 1·24 0·19
123·980
*For proposal not on award list, sums are requests not awards. †Populations at risk derived from climate-driven malaria-risk models,6 interpolated population distributions,7 and national 2000 population estimates.8 Populations exposed to risk include areas with acutely seasonal or epidemic risks through to intense stable transmission.
Four countries outside Africa (China, Indonesia, Laos, and Sri Lanka), which bear only 3% of the world’s malaria burden, would have access to 44% of the malaria funds awarded in this first cycle of funding. Support per person for antimalaria actions approved by GFATM remains paltry given the needs of Africa. Many national proposals for prevention and control received no support (panel). Proposals endorsed for funding by the Board will provide on average US$0·17 per person at risk during the first year. At this level, the targets agreed at Abuja have no chance of being met. The current cautionary policy of the Global Fund, which impels it to invest mainly in projects that pursue limited goals, will impede national RBM efforts. Malaria cannot afford to remain a poor cousin to the other diseases of poverty within GFATM’s funding process. The international community cannot afford to let the Fund become another missed opportunity to make a difference. *A Teklehaimanot, R W Snow *Center for Global Health and Economic Development, Columbia University, New York, NY 10002, USA; KEMRI/Wellcome Trust Collaborative Program, Nairobi, Kenya; and Centre for Tropical Medicine, University of Oxford, UK (e-mail: [email protected])
THE LANCET • Vol 360 • September 21, 2002 • www.thelancet.com
Epidemiology on trial—confessions of an expert witness A recent case in the UK High Court judged a claim by users of the third-generation oral contraceptive pill (3GOC), who alleged that the drug had caused a venous thromboembolism.1 I was an expert witness for the claimants, arguing for a scientific consensus.2 The judgment pronounced at two levels. Overall, the claimants lost. The judgment concluded that 3GOCs are not associated with a higher risk of venous thromboembolism than second-generation pills (2GOCs). This judgment rested on one argument3 and the judge therefore added a fallback position, which may be about right quantitatively: the relative risk of 3GOCs compared with 2GOCs for venous thromboembolism is around 1·7, and causal. Nevertheless, this relative risk was still not deemed sufficient to accept the claim, because all parties had pre-agreed that the relative risk had to be higher than 2·0 for it to be more likely than not that, for women who were harmed, the venous thromboembolism was due to use of a 3GOC. The evidence produced for no excess risk at all for 3GOCs was inconclusive, but it impressed the judge. A 1999 report3 purported to reduce a relative risk of 1·5 from the Transnational Study4 to 0·8 by appropriate adjustment for time-dependent variables with Cox proportional hazards modelling.5 The idea was to eliminate bias if women who are susceptible to venous thromboembolism changed to the newer oral contraceptives more readily, or succumb early in their first use of such a drug. Few understood the report at the time because detail was lacking, and relative risks rarely change direction by adjustment. After the analyses were made available, the claimants’ epidemiologist Alec Walker could explain with clarity that the variables used for adjustment were scientifically illegitimate. In the conversion from a case-control study (which the Transnational was) to a cohort study (which Cox’s regression requires), design constraints were ignored and discussion of the nature of a risk set, defined in all textbooks, was misrepresented. The adjusting variables related to the time of recruitment, and were current-use patterns of oral contraceptives, but used retrospectively. Using variables to adjust for spontaneous timedependent risk (in 1993, for example) measured in future time (1995 possibly) amounts to clairvoyance. Proper adjustment made little difference to the risk estimates, and could better have been done with 889
For personal use. Only reproduce with permission from The Lancet Publishing Group.
COMMENTARY
Meta-analyses of studies of combined oral contraceptives and venous thromboembolism* Study
Year
Adjusted 95% CI odds ratio
Relative weight
All studies Bloemenkamp WHO Transnational Farmer, Mediplus Herings, all users Parkin Jick, GPRD Lidegaard Pooled estimate
1995 1995 1996 1999 1999 2000 2000 2002
2·20 2·70 1·73 1·35 2·30 2·92 2·20 1·40 1·89
0·90–5·40 1·60–4·60 1·07–2·79 0·74–2·39 1·50–3·70 0·78–12·0 1·50–3·30 1·00–1·90 1·59–2·25
5 14 17 11 21 2 26 34 131
Industry-funded studies Transnational 1996 Farmer, Mediplus 1999 Farmer, GPRD 2000 Lidegaard 2002 Pooled estimate
1·73 1·35 1·20 1·40 1·35
1·07–2·79 0·74–2·39 0·90–1·70 1·00–1·90 1·12–1·63
17 11 46 34 108
Independent studies Bloemenkamp 1995 WHO 1995 Herings, all users 1999 Parkin 2000 Jick GPRD 2000 Pooled estimate
2·20 2·70 2·30 2·92 2·20 2·27
0·90–5·40 1·60–4·60 1·50–3·70 0·78–12·0 1·50–3·30 1·72–3·00
5 14 21 2 26 68
*Fixed-effect modelling. References for these studies can be derived from ref 8 except where actual results quoted were amended in Court1 to make them more mutually consistent. Exception is Lidegaard et al, 2002.7
nested case-control study. Jick and colleagues10 used the GPRD in the same way, but these analyses yield statistically different estimates (p<0·02). Farmer’s analysis is excluded overall, since it duplicates that of Jick. Wyeth had also undertaken an analysis of the GPRD,11 which was contested. This analysis remains unpublished and as such the judge deemed it irrelevant as evidence. So much for publication bias.12 Pharmacovigilance has to rely as much on research investment to refute an uncomfortable association, as on a search towards a reasonable scientific consensus.13 The actual biases that could explain the highly significant differences in source of funding remain unresolved, since all true details of execution are never reported. The report of Lewis and colleagues3 demonstrates that peer review guarantees little. The judge had the power to determine beyond doubt whether this analysis3 was correct. But one expert each was to be examined on each substantive question, which prevented discussion once the main expert for the claimants had returned to the USA and the defendant’s expert, asked to do more analyses, had died. The outcome is an unhappy one—appropriate safeguards for consumers of these products demand that a scientific consensual determination of risk is reached. The actual relative risk could still be over 2, since relative quality is ignored herein. The legal system in this case was unable to grapple with the scientific issues. Despite potential shortcomings in the claimants’ case, this trial and debate was a lost opportunity for pharmacoepidemiology. I have claimed fees from the claimants’ lawyers. The judge was critical of parts of my evidence. I am a member of the UK Committee on Safety of Medicines.
Klim McPherson conventional analyses of the original data, just as the WHO study did.6 Despite the fallback relative risk of 1·7 probably being about right, the judgment was wrong. The threshold of a twofold relative risk was incorrectly treated as representing the balance of probability argument. However, when comparing women on different choices of oral contraceptive, baseline risk among non-users has to be included properly. The balance of probability argument depends on absolute, not relative, risks; the risk of 2GOCs is already around three times baseline. A twofold relative risk requires the extra risk of using a 3GOC to exceed both the extra risk of 2GOC use plus the common baseline risk. This case should therefore have been won on its fallback assessment, with a greater extra absolute risk for 3GOCs than for 2GOCs, which is achieved with a relative risk larger than 1·67. In the Court, acceptable common characteristics for relevant studies changed according to an emergent consensus, and the amended results of all major studies agreed (panel). A revised report that appeared during the hearing7 is also included. The weights are calculated from the width of the confidence intervals. A woman on a 3GOC with a venous thromboembolism (relative risk 3GOC/2GOC, 1·89) has, on the balance of probability, had her venous thromboembolism more likely than not because of that oral contraceptive. The difference8 in estimates by source of funding is the major contribution to the heterogeneity between these results, here p<0·001. Two studies by Farmer and colleagues are included, one of which uses data from the General Practice Research Database (GPRD)9 for a 890
Department of Social Medicine, University of Bristol, Canynge Hall, Bristol BS8 2PR, UK (e-mail: [email protected])
1
2 3
4
5 6 7
8
9
10
11 12 13
High Courts of Justice. Case number 0002638, between XYZ & Others and Schering Health Care Limited, Organon Laboratories Limited and John Wyeth & Borther Limited. Approved Judgement, July 29, 2002. http://www.courtservice.gov.uk/judgmentsfiles/j1298/ xyz_-v-schering.htm (accessed Sept 17, 2002). Editorial. Epidemiology on trial. Lancet 2002; 360: 421. Lewis MA, MacRae KD, Kuhl-Habich DK, et al. The differential risk of OC’s; impact of full exposure history. Hum Reprod 1999; 14: 1493–199. Spitzer WO, Lewis MA, Heineman LA et al. Third generation oral contraceptives and risk of VTE: international case control study. BMJ 1996; 312: 83–87. Cox DR, Oakes D. Analysis of survival data. New York: Wiley, 1984. Poulter NR, Farley TM, Chang CL, et al. Safety of combined oral contraceptive pills. Lancet 1996; 347: 547. Lidegaard O, Edstrom B, Kreiner S. Oral contraceptives and venous thromboembolism: a five-year national case control study. Contraception 2002; 65: 187–96. Kemmeren JM, Algra A, Grobbee DE. Third generation oral contraceptives and risk of venous thrombosis: meta-analysis. BMJ 2001; 323: 131–34. Farmer RD, Williams TJ, Simpson EL, et al. Effect of 1995 pill scare on rates of venous thromboembolism among women taking combined oral contraceptives: analysis of general practice research database. BMJ 2000; 321: 477–79. Jick H, Kaye JA, Vasilakis-Scaramozza C, et al. Risk of VTE before and after 1995: cohort and case control study. BMJ 2000; 321: 1190–95. Brock P. Wyeth responds to news story on oral contraceptives and DVT. BMJ 2001; 322: 1605. Skegg D. Pitfalls of pharmacoepidemiology. BMJ 2000; 321: 1171–72. World Health Organization. Cardiovascular disease and steroid hormone contraception. Report of WHO scientific group. WHO technical reports series 877. Geneva: WHO, 1998.
THE LANCET • Vol 360 • September 21, 2002 • www.thelancet.com
For personal use. Only reproduce with permission from The Lancet Publishing Group.