- Email: [email protected]
A Fallacy of Intention-to-Treat Analysis
May 1991
tumours and peritoneal carcinomatosis")' the A-GRAD was found to be > 1.1 g/dL as a feature of portal hypertension. With these results we conclude that the A-GRAD is useful in the prediction of portal hypertension not only in ascitic patients with cirrhosis, peritoneal carcinomatosis, or peritoneal tuberculosis but also in atypical patients who have portal hypertension and high AF total protein. Rector (6) stated that wide A-GRAD may be misleading in several situations, such as heart failure, portal vein thrombosis, and cirrhosis complicated by peritoneal tuberculosis. On the contrary, we believe that this finding should be interpreted as an important advantage of the A-GRAD estimation. If it is thought of as a marker of portal hypertension, it will indicate the need to search for portal hypertension in such unusual cases as our mixed cases. Finally, we agree with Albillos et al. and Hoefs that A-GRAD estimation should be routinely placed in the analysis of ascites. YILMAZ CAKALOGLU. M.D. A TILLA OKTEN, M.D. SULEYMAN YALCIN, M.D.
Division of Gastroentero-Hepatology Department of Internal Medicine Istanbul Medical Faculty of Istanbul University Capa, Topkapi34390 Istanbul, Turkey 1. Albillos A, Cuervas-Mons V, Millan I, Canton T, Montes J, Barrios C, Garrido A, Escartin P. Ascitic fluid polymorphonuclear cell count and serum to ascites albumin gradient in the diagnosis of bacterial peritonitis. Gastroenterology 1990;98:13440. 2. Hoefs JC: Diagnostic aracentesis: a potent clinical tool. Gastroenterology 1990;98:230-236. 3. Hoefs JC: Serum protein concentration and portal pressure determine the ascitic fluid protein concentration in patients with chronic liver disease. J Lab Clin Med 1983;102:261-273. 4. Pare P, Talbot J, Hoefs JC: Serum-ascites albumin concentration gradient: a physiologic approach to the differential diagnosis of ascites. Gastroenterology 1983;85:240-244. 5. Rector WG, Reynolds TB: Superiority of the serum-ascites albumin difference over the ascites total protein concentration in separation of "transudative" and "exudative" ascites. Am J Med 1984;77:83-85. 6. Rector WG: An improved diagnostic approach to ascites. Arch Intern Med 1987;147:215. Reply. Cakaloglu et al. report the value of the A-GRAD in the clinical approach to the ascitic patient and emphasize the particular value of this approach in patients with high protein ascites. We certainly agree. Furthermore, only 4 of 28 (14.3%) patients with another cause for ascites were found to have the coincidental presence of portal hypertension. Thus, reanalysis of their data along more traditional lines in which a wide A-GRAD in patients with other forms of ascites is treated as a false-pOSitive yield an accuracy for total protein of 87.7%, for ascites/serum total protein ratio of 93.4%, and for the A-GRAD of 95.3%. Even from a traditional viewpoint, the results of those four patients did not interfere with the value of the A-GRAD compared with other tests. This is similar to the data of Albillos et al. (1). However, I in agree with the authors that although these four patients represent false-positive data points for an etiologic causation, they represent true positives for the presence of portal hypertenSion. Given an individual clinical context, I have always found a knowledge of the presence of portal hypertenSion valuable and rarely confusing. However, maximum use of the A-GRAD requires integration of the information of the presence of portal hypertension with other
CORRESPONDENCE
1485
clinical data and the rest of the ascitic analysis (2). This is a rewarding and not unpleasant task for the thinking physician. JOHN C. HOEFS, M.D.
Division of Gastroenterology Department of Medicine University of California Irvine, California 92717 1. Albillos A, Cuervas-Mons V, Millan I, Cant6n T, Montes J, Barrios C, Garrido A, Escartin P. Ascitic fluid polymorphonuclear cell count and serum to ascites albumin gradient in the diagnosis of bacterial peritonitis. Gastroenterology 1990;98:13440. 2. Hoefs JC: Diagnosis paracentesis: a potent clinical tool. Gastroenterology 1990;98:230-236.
A Fallacy of Intention-to-Treat Analysis Dear Sir: We would appreciate the opportunity to comment on the article by Pym et al. (1) regarding the cost-effectiveness of cimetidine for the long-term maintenance treatment of patients with peptic ulcer disease. Coming as it does from a center renowned for its contributions to research into ulcer disease, the study will undoubtedly be very influential. It seems to us that the conclusions of the study may have been biased by the methodology used to analyze the data. Our attention was drawn to this problem by the high rate of ulcer complications recorded by Pym et al. In patients with gastric ulcers, 7 of 77 (9.1 %) receiving maintenance treatment with cimetidine and 2 of 75 (2.6%) receiving placebo developed hemorrhage during 3 years of follow-up while, during a similar observation period in patients with duodenal ulcers, 2 of 51 (3.8%) receiving maintenance treatment with cimetidine and 2 of 58 patients (3.4%) receiving placebo developed hemorrhage. From these results, it appears that maintenance treatment with cimetidine confers no protection against the complications of ulcer disease. Moreover, the results differ markedly from preViously reported low rates of hemorrhage during long-term maintenance treatment with cimetidine (2,3). In our studies with ranitidine, the cumulative risk after 6 years of continuous maintenance treatment was only 1.3% in patients with duodenal ulcers (4) and zero in patients with gastric ulcers (5). Indeed, maintenance treatment with ranitidine conferred a highly significant and > 10-fold protection against ulcer hemorrhage compared with no active antiulcer therapy. It seems possible that the exceptionally high rates of complications reported by Pym et al. depend on the method of statistical analysis used in the latter study. Pym et al. used an "intention-totreat" analysis-a technique that considers that any patient allocated to a defined treatment group must be analyzed as having continuously received that treatment throughout the period of observation. Unfortunately, some or all of the patients studied by Pym et al. who were classified as having developed hemorrhage during maintenance treatment with cimetidine might, in fact, not have been receiving maintenance treatment for some time before the development of the complication of ulcer recurrence. In this connection, we have reexamined our raw data relating to long-term maintenance treatment of patients with duodenal ulcer disease (4). If a standard intention-to-treat analysis had been performed, 10 patients would have been classified as having developed hemorrhage at the time of ulcer recurrence while receiving ranitidine, although 6 of the 10 patients had stopped maintenance treatment before the occurrence of hemorrhage; therefore, only 4 of the 10 patients actually developed hemorrhage during maintenance treatment with ranitidine. In addition, when we considered the 8
1486
CORRESPONDENCE
GASTROENTEROLOGY Vol. 100, No.5
patients who had developed hemorrhage associated with ulcer recurrence during periods without antiulcer therapy, we found that 7 of these patients had developed hemorrhage after the maintenance treatment period had ended, It seems clear that great care is required in interpreting the results of intention-to-treat analyses (6). It is particularly important to note that although a patient may be classified as belonging to the "active treatment" group for the purpose of such analysis, this categorization unfortunately does not mean that the patient is actually receiving active treatment at the time of ulcer recurrence or complication. In this connection, it is also worth emphasizing that ulcer recurrence and complications may develop very rapidly when active antiulcer treatment is stopped (7,8). Consequently, intentionto-treat analysis cannot provide an accurate estimate of the real rates of ulcer recurrence or complications during continuous prolonged antiulcer therapy. To obtain true rates of complicated or uncomplicated symptomatic ulcer recurrence during long-term maintenance treatment on which to base cost-benefit analyses, other analytical methods must be used (9).
J. G. PENSTON K. G. WORMS LEY
Ninewells Hospital and Medical School Ninewells Dundee DD1 9SY, Scotland 1. Pym B, Sandstad J, Seville P, Byth K, Middleton WRJ, Talley NJ,
2.
3.
4.
5.
Piper OW. Cost-effectiveness of cimetidine maintenance therapy in chronic gastric and duodenal ulcer. Gastroenterology 1990;99: 27-35. Walan A, Bianchi Porro G, Hentschel E, Bardhan KD, Delattre M. Maintenance treatment with cimetidine in peptic ulcer disease for up to 4 years. Scand J GastroenteroI1987;22:397-405. Bardhan KD. Six years of continuous cimetidine treatment in peptic ulcer disease: efficacy and safety. Aliment Pharmacol Ther 1988;2:395-405. Pens ton JG, Wormsley KG. Efficacy and safety of long-term maintenance treatment of duodenal ulcers. Scand J GastroenteroI1989;24:1145-1152. Pens ton JG, Worms ley KG. Long-term maintenance treatment of gastric ulcers with ranitidine. Aliment Pharmacol Ther 1990;4: 339-355.
6. Feinstein AR. An additional basic science for clinical medicine: II. The limitations of randomized trials. Ann Intern Med 1983;99: 544-550. 7. Saunders JHB, Wormsley KG. Long-term effects and after-effects of treatment of duodenal ulcer with metiamide. Lancet 1977;1: 765-767. 8. Wallace WA, Orr CME, Beam AR. Perforation of chronic peptic ulcers after cimetidine. Br Med J 1977;2:865-866. 9. Penston JG. Efficacy and safety of long-term maintenance treatment of duodenal ulcers. Scand J Gastroenterol1990;25 (Suppl): (in press). Reply. We thank Drs. Penston and Worms ley for their constructive comments on our paper (1), which are very pertinent to our study. First, they question the use of the intent-to-treat principle in our analysis. We emphasize that our study relates primarily to the cost-effectiveness of treatment in a double-blind, placebo-controlled study, rather than to the efficacy of cimetidine, which is already widely acknowledged. It was therefore essential to use a model applicable to conventional medical practice. A working party set up by the Biopharmaceutical Section of the American Statistical Association (2) has emphasized that physicians de facto act under the intention-to-treat principle and that this most closely models the clinical scenario. Also, Armitage (3) has recommended that the primary analysis of a randomized clinical trial should follow the intention-to-treat principle. Both authorities state that after the initial intent-to-treat analysis, secondary analyses may be performed, restricting attention to those who comply with their therapeutic assignment. Such was done in our study, in which we studied the responses of the compliant patients only and showed essentially the same results, as noted in our report. Our method of analysis was identical to that used by Penston and Worms ley in their study of long-term maintenance treatment of gastric ulcer, in which they used intention-to-treat and efficacy methods (4). Drs. Pens ton and Worms ley comment on our findings regarding the frequency of bleeding in the follow-up of healed duodenal ulcer (DU) and gastric ulcer (GU) patients and question whether the high frequency of bleeding in our series might be due to protocol violation, patients' ceasing treatment and subsequently bleeding. Factors that may influence frequency of observed bleeding during
Table 1. All Patients Who Completed 3 Years of Study Gastric ulcer Cimetidine
Before trial Intention to treat Bleeding No bleeding Compliant patients only Bleeding No bleeding Noncompliant patients Bleeding No bleeding
No. who bled during trial (% of total)
Duodenal ulcer Placebo
Total
3 (14.3%) 4 (7.1%)
21
2 (12.5%) 3 (8.6%)
16
1 (20%) 1 (4.8%)
5
56
35
21
No. who bled during trial (%) of total) 1 (5.6%) 1 (1.8%) 1 (11.1%) 0 (0%) 0 (0%) 1 (3.3%)
Cimetidine
Total 18 57
9 27
9 30
No. who bled during trial (% of total) 0 (0%) 2 (5.3%) 0 (0%) 1 (4.3%) 0 (0%) 1 (6.7%)
Placebo
Total 13 38
9 23
4 15
No. who bled during trial (% of total) 1 (7.1%) 1 (2.3%) 1 (14.3%) 1 (6.7%) 0 (0%) 0 (0%)
Total 14 44
7 15
7 29
tumours and peritoneal carcinomatosis")' the A-GRAD was found to be > 1.1 g/dL as a feature of portal hypertension. With these results we conclude that the A-GRAD is useful in the prediction of portal hypertension not only in ascitic patients with cirrhosis, peritoneal carcinomatosis, or peritoneal tuberculosis but also in atypical patients who have portal hypertension and high AF total protein. Rector (6) stated that wide A-GRAD may be misleading in several situations, such as heart failure, portal vein thrombosis, and cirrhosis complicated by peritoneal tuberculosis. On the contrary, we believe that this finding should be interpreted as an important advantage of the A-GRAD estimation. If it is thought of as a marker of portal hypertension, it will indicate the need to search for portal hypertension in such unusual cases as our mixed cases. Finally, we agree with Albillos et al. and Hoefs that A-GRAD estimation should be routinely placed in the analysis of ascites. YILMAZ CAKALOGLU. M.D. A TILLA OKTEN, M.D. SULEYMAN YALCIN, M.D.
Division of Gastroentero-Hepatology Department of Internal Medicine Istanbul Medical Faculty of Istanbul University Capa, Topkapi34390 Istanbul, Turkey 1. Albillos A, Cuervas-Mons V, Millan I, Canton T, Montes J, Barrios C, Garrido A, Escartin P. Ascitic fluid polymorphonuclear cell count and serum to ascites albumin gradient in the diagnosis of bacterial peritonitis. Gastroenterology 1990;98:13440. 2. Hoefs JC: Diagnostic aracentesis: a potent clinical tool. Gastroenterology 1990;98:230-236. 3. Hoefs JC: Serum protein concentration and portal pressure determine the ascitic fluid protein concentration in patients with chronic liver disease. J Lab Clin Med 1983;102:261-273. 4. Pare P, Talbot J, Hoefs JC: Serum-ascites albumin concentration gradient: a physiologic approach to the differential diagnosis of ascites. Gastroenterology 1983;85:240-244. 5. Rector WG, Reynolds TB: Superiority of the serum-ascites albumin difference over the ascites total protein concentration in separation of "transudative" and "exudative" ascites. Am J Med 1984;77:83-85. 6. Rector WG: An improved diagnostic approach to ascites. Arch Intern Med 1987;147:215. Reply. Cakaloglu et al. report the value of the A-GRAD in the clinical approach to the ascitic patient and emphasize the particular value of this approach in patients with high protein ascites. We certainly agree. Furthermore, only 4 of 28 (14.3%) patients with another cause for ascites were found to have the coincidental presence of portal hypertension. Thus, reanalysis of their data along more traditional lines in which a wide A-GRAD in patients with other forms of ascites is treated as a false-pOSitive yield an accuracy for total protein of 87.7%, for ascites/serum total protein ratio of 93.4%, and for the A-GRAD of 95.3%. Even from a traditional viewpoint, the results of those four patients did not interfere with the value of the A-GRAD compared with other tests. This is similar to the data of Albillos et al. (1). However, I in agree with the authors that although these four patients represent false-positive data points for an etiologic causation, they represent true positives for the presence of portal hypertenSion. Given an individual clinical context, I have always found a knowledge of the presence of portal hypertenSion valuable and rarely confusing. However, maximum use of the A-GRAD requires integration of the information of the presence of portal hypertension with other
CORRESPONDENCE
1485
clinical data and the rest of the ascitic analysis (2). This is a rewarding and not unpleasant task for the thinking physician. JOHN C. HOEFS, M.D.
Division of Gastroenterology Department of Medicine University of California Irvine, California 92717 1. Albillos A, Cuervas-Mons V, Millan I, Cant6n T, Montes J, Barrios C, Garrido A, Escartin P. Ascitic fluid polymorphonuclear cell count and serum to ascites albumin gradient in the diagnosis of bacterial peritonitis. Gastroenterology 1990;98:13440. 2. Hoefs JC: Diagnosis paracentesis: a potent clinical tool. Gastroenterology 1990;98:230-236.
A Fallacy of Intention-to-Treat Analysis Dear Sir: We would appreciate the opportunity to comment on the article by Pym et al. (1) regarding the cost-effectiveness of cimetidine for the long-term maintenance treatment of patients with peptic ulcer disease. Coming as it does from a center renowned for its contributions to research into ulcer disease, the study will undoubtedly be very influential. It seems to us that the conclusions of the study may have been biased by the methodology used to analyze the data. Our attention was drawn to this problem by the high rate of ulcer complications recorded by Pym et al. In patients with gastric ulcers, 7 of 77 (9.1 %) receiving maintenance treatment with cimetidine and 2 of 75 (2.6%) receiving placebo developed hemorrhage during 3 years of follow-up while, during a similar observation period in patients with duodenal ulcers, 2 of 51 (3.8%) receiving maintenance treatment with cimetidine and 2 of 58 patients (3.4%) receiving placebo developed hemorrhage. From these results, it appears that maintenance treatment with cimetidine confers no protection against the complications of ulcer disease. Moreover, the results differ markedly from preViously reported low rates of hemorrhage during long-term maintenance treatment with cimetidine (2,3). In our studies with ranitidine, the cumulative risk after 6 years of continuous maintenance treatment was only 1.3% in patients with duodenal ulcers (4) and zero in patients with gastric ulcers (5). Indeed, maintenance treatment with ranitidine conferred a highly significant and > 10-fold protection against ulcer hemorrhage compared with no active antiulcer therapy. It seems possible that the exceptionally high rates of complications reported by Pym et al. depend on the method of statistical analysis used in the latter study. Pym et al. used an "intention-totreat" analysis-a technique that considers that any patient allocated to a defined treatment group must be analyzed as having continuously received that treatment throughout the period of observation. Unfortunately, some or all of the patients studied by Pym et al. who were classified as having developed hemorrhage during maintenance treatment with cimetidine might, in fact, not have been receiving maintenance treatment for some time before the development of the complication of ulcer recurrence. In this connection, we have reexamined our raw data relating to long-term maintenance treatment of patients with duodenal ulcer disease (4). If a standard intention-to-treat analysis had been performed, 10 patients would have been classified as having developed hemorrhage at the time of ulcer recurrence while receiving ranitidine, although 6 of the 10 patients had stopped maintenance treatment before the occurrence of hemorrhage; therefore, only 4 of the 10 patients actually developed hemorrhage during maintenance treatment with ranitidine. In addition, when we considered the 8
1486
CORRESPONDENCE
GASTROENTEROLOGY Vol. 100, No.5
patients who had developed hemorrhage associated with ulcer recurrence during periods without antiulcer therapy, we found that 7 of these patients had developed hemorrhage after the maintenance treatment period had ended, It seems clear that great care is required in interpreting the results of intention-to-treat analyses (6). It is particularly important to note that although a patient may be classified as belonging to the "active treatment" group for the purpose of such analysis, this categorization unfortunately does not mean that the patient is actually receiving active treatment at the time of ulcer recurrence or complication. In this connection, it is also worth emphasizing that ulcer recurrence and complications may develop very rapidly when active antiulcer treatment is stopped (7,8). Consequently, intentionto-treat analysis cannot provide an accurate estimate of the real rates of ulcer recurrence or complications during continuous prolonged antiulcer therapy. To obtain true rates of complicated or uncomplicated symptomatic ulcer recurrence during long-term maintenance treatment on which to base cost-benefit analyses, other analytical methods must be used (9).
J. G. PENSTON K. G. WORMS LEY
Ninewells Hospital and Medical School Ninewells Dundee DD1 9SY, Scotland 1. Pym B, Sandstad J, Seville P, Byth K, Middleton WRJ, Talley NJ,
2.
3.
4.
5.
Piper OW. Cost-effectiveness of cimetidine maintenance therapy in chronic gastric and duodenal ulcer. Gastroenterology 1990;99: 27-35. Walan A, Bianchi Porro G, Hentschel E, Bardhan KD, Delattre M. Maintenance treatment with cimetidine in peptic ulcer disease for up to 4 years. Scand J GastroenteroI1987;22:397-405. Bardhan KD. Six years of continuous cimetidine treatment in peptic ulcer disease: efficacy and safety. Aliment Pharmacol Ther 1988;2:395-405. Pens ton JG, Wormsley KG. Efficacy and safety of long-term maintenance treatment of duodenal ulcers. Scand J GastroenteroI1989;24:1145-1152. Pens ton JG, Worms ley KG. Long-term maintenance treatment of gastric ulcers with ranitidine. Aliment Pharmacol Ther 1990;4: 339-355.
6. Feinstein AR. An additional basic science for clinical medicine: II. The limitations of randomized trials. Ann Intern Med 1983;99: 544-550. 7. Saunders JHB, Wormsley KG. Long-term effects and after-effects of treatment of duodenal ulcer with metiamide. Lancet 1977;1: 765-767. 8. Wallace WA, Orr CME, Beam AR. Perforation of chronic peptic ulcers after cimetidine. Br Med J 1977;2:865-866. 9. Penston JG. Efficacy and safety of long-term maintenance treatment of duodenal ulcers. Scand J Gastroenterol1990;25 (Suppl): (in press). Reply. We thank Drs. Penston and Worms ley for their constructive comments on our paper (1), which are very pertinent to our study. First, they question the use of the intent-to-treat principle in our analysis. We emphasize that our study relates primarily to the cost-effectiveness of treatment in a double-blind, placebo-controlled study, rather than to the efficacy of cimetidine, which is already widely acknowledged. It was therefore essential to use a model applicable to conventional medical practice. A working party set up by the Biopharmaceutical Section of the American Statistical Association (2) has emphasized that physicians de facto act under the intention-to-treat principle and that this most closely models the clinical scenario. Also, Armitage (3) has recommended that the primary analysis of a randomized clinical trial should follow the intention-to-treat principle. Both authorities state that after the initial intent-to-treat analysis, secondary analyses may be performed, restricting attention to those who comply with their therapeutic assignment. Such was done in our study, in which we studied the responses of the compliant patients only and showed essentially the same results, as noted in our report. Our method of analysis was identical to that used by Penston and Worms ley in their study of long-term maintenance treatment of gastric ulcer, in which they used intention-to-treat and efficacy methods (4). Drs. Pens ton and Worms ley comment on our findings regarding the frequency of bleeding in the follow-up of healed duodenal ulcer (DU) and gastric ulcer (GU) patients and question whether the high frequency of bleeding in our series might be due to protocol violation, patients' ceasing treatment and subsequently bleeding. Factors that may influence frequency of observed bleeding during
Table 1. All Patients Who Completed 3 Years of Study Gastric ulcer Cimetidine
Before trial Intention to treat Bleeding No bleeding Compliant patients only Bleeding No bleeding Noncompliant patients Bleeding No bleeding
No. who bled during trial (% of total)
Duodenal ulcer Placebo
Total
3 (14.3%) 4 (7.1%)
21
2 (12.5%) 3 (8.6%)
16
1 (20%) 1 (4.8%)
5
56
35
21
No. who bled during trial (%) of total) 1 (5.6%) 1 (1.8%) 1 (11.1%) 0 (0%) 0 (0%) 1 (3.3%)
Cimetidine
Total 18 57
9 27
9 30
No. who bled during trial (% of total) 0 (0%) 2 (5.3%) 0 (0%) 1 (4.3%) 0 (0%) 1 (6.7%)
Placebo
Total 13 38
9 23
4 15
No. who bled during trial (% of total) 1 (7.1%) 1 (2.3%) 1 (14.3%) 1 (6.7%) 0 (0%) 0 (0%)
Total 14 44
7 15
7 29