- Email: [email protected]
Clinical trials and data integrity
Clinical Trials and Data Integrity
Testimony before a s u b c o m m i t t e e of the United States Senate last year [1] coupled with the recent series of articles in The Boston Globe [2,3] on falsification of data in one of the clinics of the Eastern Cooperative O n c o l o g y G r o u p (ECOG) should r e m i n d us all of the need for constant vigilance to protect against such occurrences. Certainly integrity p r o b l e m s are not u n i q u e to clinical trials, as witnessed by two separate accounts of plagiarism in the scientific c o m m u n i t y in the last few m o n t h s [4-6]. Nor are such problems necessarily any more frequent in the multicenter trial than in the smaller scale single-center trial, but the likelihood of detection may be greater since most large-scale trials have dedicated data centers to receive and edit the data as they are generated. The smaller trial may not be able to afford this degree of svecialization. O n g o i n g editing procedures, while designed primarily to detect errors in the data collection process, may detect fradulent data as well. For example, comparison of the within patientexamination variability of blood pressure readings was instrumental in identifying forged second readings in the Lipid Research Clinics [7]. In a d d i t i o n to such editing p r o c e d u r e s , m a n y large-scale trials carry out periodic visits to participating clinics to monitor the quality of the data collection and recording process. Some studies, such as the P e r s a n t i n e Aspirin Reinfarction S t u d y (PARIS) [8[ and the National Cooperative Gallstone Study (NCGS) [9] have extended the principle to include the data coordinating center as well. The Food and Drug Administration (FDA) has assumed an increasingly active role in auditing trials which are carried out with their approval [10-12]. The vigilance against falsification is especially important in clinical trials since the normal rule of science, requiring replication before a result is accepted, does not always apply. I n d e p e n d e n t replication of trials that have taken years and millions of dollars to carry out may be impossible to repeat. The integrity of a data set is a function of the entire data collection and analysis process, H o n e s t y in data analysis and reporting is as i m p o r t a n t as integrity in the data g e n e r a t i o n process. Discarding selected " o u t l i e r " laboratory values or data from patients w h o did not take the s t u d y treatment, even if d o n e without malice of forethought, may in fact have a far more serious impact on the conclusions of the study than will a few minor isolated cases of data falsification in a participating clinic. Often questions regarding a s t u d y will center a r o u n d the way in which o u t c o m e events are counted and the d e n o m i n a t o r s used for c o m p a r i s o n of the s t u d y groups. For example, legitimate questions have been raised regarding the Anturane-Reinfarction Trial (ART) d u e to failure to count some events because of m i n o r protocol violations [13]. D o u b t s can also arise because of the n a t u r e of the r e l a t i o n s h i p of Controlled Clinical Trials, 1, 189-192 (1980) © 1980 Elsevier North Holland, Inc., 52 Vanderbilt Avenue, New York, NY 10017
189 0197-2456/8~'030189004502.25
190
Editorial investigators to the m a n u f a c t u r e r of the p r o d u c t b e i n g tested. The NCGS requires its investigators and o v e r s i g h t c o m m i t t e e s to disclose conflicts of interest so that they m a y be r e v i e w e d and, w h e r e necessary, acted u p o n even though the s t u d y receives no i n d u s t r y s u p p o r t from the m a n u f a c t u r e r of the d r u g b e i n g tested. The I'ARIS, f u n d e d by B o e h r i n g e r - l n g e l h e i m , p r o h i b i t e d investigators from h a v i n g any financial relationship with the m a n u f a c t u r e r of Persantine [81. The failure to p r o v i d e a clear separation of those r e s p o n s i b l e for carrying out the s t u d y from the s p o n s o r i n g agency, in the case of i n d u s t r y s u p p o r t e d trials, can raise d o u b t s r e g a r d i n g the credibility of the study. The ART has b e e n o'iticised b e c a u s e the data c o o r d i n a t i n g center was o p e r a t e d by the m a n u f a c t u r e r of A n t u r a n e [131. It is difficult to dispell the d o u b t s such relationships create regardless of w h e t h e r or not they are likely to result in any data or analysis questions. P r i m a r y e m p h a s i s m u s t be placed on p r e v e n t i o n of q u e s t i o n a b l e data practices rather than on detection of p r o b l e m s after the fact, since there is no foolproof w a y to p r e v e n t falsification of data. It is worth noting that the p r o b l e m in the Boston clinic did not c o m e to light t h r o u g h any routine m o n i t o r i n g carried out by the study. The foremost d e f e n s e against such p r o b l e m s lies in the r e c r u i t m e n t and retention of honest, conscientious investigators w h o are selected for their scientific m a t u r i t y and integrity and w h o in turn are able to recruit s u p p o r t p e r s o n n e l w h o m a i n t a i n high s t a n d a r d s of integrity in the data collection process. At the s a m e time, s p o n s o r s of trials m u s t be p r e p a r e d to p r o v i d e a d e q u a t e s u p p o r t for essential activities, including those p e r f o r m e d b y the data center. U n d e r f u n d i n g , w i t h o u t a c o r r e s p o n d i n g cut back in the d e m a n d s placed on a center, can lead to serious p r o b l e m s in the trial. S p o n s o r s a n d leaders of trials m u s t be ,nindful of factors that m a y c o n t r i b u t e to p r o b l e m s in the s t u d y data and a t t e m p t to m i n i m i z e their influence. Peer p r e s s u r e from within the study, while a powerful incentive for i m p r o v i n g p e r f o r m a n c e , can result in faulty data if excessive. S p o n s o r s of trials a n d the policy b o a r d s that oversee them m u s t use restraint in issuing threats of f u n d i n g cutoffs as p r o d s for i m p r o v i n g p e r f o r m a n c e . Such threats m a y lead to shortcuts in the data collection process. Even if the principal i n v e s t i g a t o r of a clinic is i m p e r v i o u s to these threats, s u p p o r t p e r s o n n e l at the clinic w h o d r a w their salary from the s t u d y m a y not be. The contract m o d e of f u n d i n g , used in place of grant s u p p o r t in recent years by the National Institutes of Health to s u p p o r t various large scale multicenter trials, has e n h a n c e d the o p p o r t u n i t i e s for "fiscal h a r a s s m e n t " and for i m p l e m e n t i n g f u n d i n g cutoffs. In a d d i t i o n , the use of p a y m e n t s c h e d u l e s with the a m o u n t p r o v i d e d scaled to the n u m b e r of p a t i e n t s enrolled a n d followed, as u s e d in so,he i n d u s t r y - s p o n s o r e d trials, m a y lead to e n r o l l m e n t of u n a c c e p t a b l e patients and to the use of coercive t e c h n i q u e s to m a i n t a i n i n d i v i d u a l s u n d e r follow-up. P r o b l e m s c o n c e r n i n g the integrity of the data .nay arise from errors or p u r p o s e f u l data m a n i p u l a t i o n . Both k i n d s of p r o b l e m s m u s t be dealt with in a forthright m a n n e r to p r e s e r v e the integrity of the study. Major errors that c a n n o t be resolved after careful r e v i e w , especially those that are treatment-related, m a y require drastic action such as that used in the ECOG s t u d y w h e r e all data from the Boston clinic were p u r g e d from the central data files of the study. The H y p e r t e n s i o n Detection and Follow-up Program
Editorial
191 (HDFP) eliminated data from one of the clinics in that s t u d y because of a systematic error, which occurred d u r i n g the r a n d o m i z a t i o n process, that could not be resolved [14]. It m a y not be necessary to p u r g e faulty data if they are part of the information collected prior to r a n d o m i z a t i o n , since they cannot be treatment related in such cases. The Multiple Risk Factor Intervention Trial (MRFIT) elected to retain a small a m o u n t of discrepant data [15] from one of its clinics, since the discrepancies arose on examinations p e r f o r m e d before patients were assigned to treatment. A claim of data manipulation that arises after the study results have been p u b l i s h e d poses special problems. Responsible m e m b e r s of the scientific c o m m u n i t y should be wary of any u n s u b s t a n t i a t e d claim of data manipulation and should avoid assuming that absence of efforts by the investigators to a n s w e r the charge provides validity for the claim. If the claim is justified, new analyses should be carried out and reported after questionable data have been eliminated. This approach is not appropriate if the claim is without validity. Investigators from the study in this case must decide w h e t h e r or not to c o m m i t r e s o u r c e s to r e s p o n d to the claim. U G D P investigators, faced with such claims, elected to r e s p o n d to criticisms a p p e a r i n g in the scientific literature, but chose to ignore those appearing in the Medical Tribune and other u n r e v i e w e d periodicals. I n d e p e n d e n t audits of the s t u d y - - o n e by a committee a p p o i n t e d by the Biometrics Society and the other by the F D A - - w e r e instrumental in dispelling the claims in this case [16,17]. The j u d g e m e n t regarding the weight to be given to any set of published results should be based on assessment of the data quality. Published papers should p r o v i d e readers with a description of the p r o c e d u r e s used d u r i n g the s t u d y to insure the integrity of the data, including an account of any purge of faulty or e r r o n e o u s data. If m e t h o d s used for quality assurance and protection of data integrity are too detailed for inclusion in the paper containing the study results, they should be made available by publication of a separate m o n o g r a p h or through some other means. Several studies have d e v e l o p e d m o n o g r a p h s that provide information on the s t u d y design and baseline results leg, 8,18-20]. Reports of this kind could easily be e x p a n d e d to include the necessary information detailing data quality and integrity p r o c e d u r e s as well. The establishment of higher reporting standards for trials would ultimately provide a more realistic framework for evaluation of trials. As it stands at present, we are still in the paradoxical situation where a trial that pays little attention to data quality control and as a result has few if any data discrepancies to report, is v i e w e d as being superior to one with excellent o n g o i n g quality control procedures, but which, because of those p r o c e d u r e s has a n u m b e r of data deficiencies to report in the publications from the study. Controlled Clinical Trials encourages the s u b m i s s i o n of w o r t h w h i l e papers that deal with data quality and integrity procedures, w h e t h e r they relate to a specific trial or to a general class of trials. Such publications will provide a better u n d e r s t a n d i n g of the data problems in clinical trials and will help investigators to design and carry out better trials in the future. Curtis L. Meinert
192
Editorial
REFERENCES 1. Holden C: FDA tells senators of doctors who fake data in clinical drug trials. Science 206:432-433, 1979. 2. Boston Sunday Globe, June 29, 1980. 3. The Boston Globe, June 30, July 1-3, 1980. 4. Broad WJ: Would be academician pirates papers. Science 208:1438-1440, 1980. 5. Broad WJ: Imbroglio at Yale (I): Emergence of a fraud. Science 210:38-41, 1980. 6. Broad WJ: Imbroglio at Yale (II): A top job lost. Science 210:171-173, 1980. 7. Schey HM, Davis CE: The use of multiple measurements to monitor protocol adherence in epidemiological studies. Department of Biostatistics, University of North Carolina, Institute of Statistics, Mimeo Series, No. 1099, December, 1976. 8. Persantine-Aspirin Reinfarction Study Research Group: Design, methods and baseline results. Circulation 62 (part II), monograph, No. 71, 1980. 9. National Cooperative Gallstone Study Coordinating Center, Cedars-Sinai Medical Center, Los Angeles, California. 10. Kelsey FO: Biomedical monitoring. Journal of Clinical Pharmacology 18:3-9, 1978. 11. Department of Health, Education and Welfare, Food and Drug Administration: Clinical investigations: Proposed establishment of regulations on obligations of sponsors and monitors. Federal Register 42(Part IV):49612-49630, September 27, 1977. 12. Department of Health, Education, and Welfare, Food and Drug Administration: Obligations of clinical investigators of regulated articles: Prepared establishment of regulations. Federal Register 47(Part 47(Part I):35210-35236, August 8, 1978. 13. Doubts about Anturane. l.ancet 2:306-307, 1980. 14. Hypertension Detection and Follow-up Program Cooperative Group: Therapeutic control of blood pressure in the hypertension detection and follow-up program cooperative group. Prev Med 8:2-13, 1979. 15. St. Louis Globe Democrat, May 3, 1976. 16. Committee for the Assessment of Biometric Aspects of Controlled Trials of Hypoglycemic Agents: Report of the committee for the assessment of biometric aspects of controlled trials of hypoglycemic agents. JAMA 231:583-608, 1975. 17. Bilstad JM, Gurian JM, Lisook AB, Litt BD, Shanahaw EJ: The Food and Drug Administration Audit of the University Group Diabetes Program. Food and Drug Administration, October 16, 1978. 18. Klimt CR, Knatterud GI,, Meinert CL, Prout TE: The University Group Diabetes Program: A study of the effects of hypoglycemic agents on vascular complications in patients with adult-onset diabetes: I. Design methods and baseline results. Diabetes 19 (suppl 2):747-782, 1970. 19. The Coronary Drug Project Research Group: The coronary drug project: Design, methods and baseline results. Circulation 47 (suppl 1): 47:1-1 to 1-50, 1973. 20. Aspirin Myocardial Infarction Study Research Group: Aspirin myocardial infarction study: Design, methods and baseline results. U.S. Department of Health and Human Services, Public Health Service, National Institutes of Health, NIH Publication No. 80-2106, 198{).
Testimony before a s u b c o m m i t t e e of the United States Senate last year [1] coupled with the recent series of articles in The Boston Globe [2,3] on falsification of data in one of the clinics of the Eastern Cooperative O n c o l o g y G r o u p (ECOG) should r e m i n d us all of the need for constant vigilance to protect against such occurrences. Certainly integrity p r o b l e m s are not u n i q u e to clinical trials, as witnessed by two separate accounts of plagiarism in the scientific c o m m u n i t y in the last few m o n t h s [4-6]. Nor are such problems necessarily any more frequent in the multicenter trial than in the smaller scale single-center trial, but the likelihood of detection may be greater since most large-scale trials have dedicated data centers to receive and edit the data as they are generated. The smaller trial may not be able to afford this degree of svecialization. O n g o i n g editing procedures, while designed primarily to detect errors in the data collection process, may detect fradulent data as well. For example, comparison of the within patientexamination variability of blood pressure readings was instrumental in identifying forged second readings in the Lipid Research Clinics [7]. In a d d i t i o n to such editing p r o c e d u r e s , m a n y large-scale trials carry out periodic visits to participating clinics to monitor the quality of the data collection and recording process. Some studies, such as the P e r s a n t i n e Aspirin Reinfarction S t u d y (PARIS) [8[ and the National Cooperative Gallstone Study (NCGS) [9] have extended the principle to include the data coordinating center as well. The Food and Drug Administration (FDA) has assumed an increasingly active role in auditing trials which are carried out with their approval [10-12]. The vigilance against falsification is especially important in clinical trials since the normal rule of science, requiring replication before a result is accepted, does not always apply. I n d e p e n d e n t replication of trials that have taken years and millions of dollars to carry out may be impossible to repeat. The integrity of a data set is a function of the entire data collection and analysis process, H o n e s t y in data analysis and reporting is as i m p o r t a n t as integrity in the data g e n e r a t i o n process. Discarding selected " o u t l i e r " laboratory values or data from patients w h o did not take the s t u d y treatment, even if d o n e without malice of forethought, may in fact have a far more serious impact on the conclusions of the study than will a few minor isolated cases of data falsification in a participating clinic. Often questions regarding a s t u d y will center a r o u n d the way in which o u t c o m e events are counted and the d e n o m i n a t o r s used for c o m p a r i s o n of the s t u d y groups. For example, legitimate questions have been raised regarding the Anturane-Reinfarction Trial (ART) d u e to failure to count some events because of m i n o r protocol violations [13]. D o u b t s can also arise because of the n a t u r e of the r e l a t i o n s h i p of Controlled Clinical Trials, 1, 189-192 (1980) © 1980 Elsevier North Holland, Inc., 52 Vanderbilt Avenue, New York, NY 10017
189 0197-2456/8~'030189004502.25
190
Editorial investigators to the m a n u f a c t u r e r of the p r o d u c t b e i n g tested. The NCGS requires its investigators and o v e r s i g h t c o m m i t t e e s to disclose conflicts of interest so that they m a y be r e v i e w e d and, w h e r e necessary, acted u p o n even though the s t u d y receives no i n d u s t r y s u p p o r t from the m a n u f a c t u r e r of the d r u g b e i n g tested. The I'ARIS, f u n d e d by B o e h r i n g e r - l n g e l h e i m , p r o h i b i t e d investigators from h a v i n g any financial relationship with the m a n u f a c t u r e r of Persantine [81. The failure to p r o v i d e a clear separation of those r e s p o n s i b l e for carrying out the s t u d y from the s p o n s o r i n g agency, in the case of i n d u s t r y s u p p o r t e d trials, can raise d o u b t s r e g a r d i n g the credibility of the study. The ART has b e e n o'iticised b e c a u s e the data c o o r d i n a t i n g center was o p e r a t e d by the m a n u f a c t u r e r of A n t u r a n e [131. It is difficult to dispell the d o u b t s such relationships create regardless of w h e t h e r or not they are likely to result in any data or analysis questions. P r i m a r y e m p h a s i s m u s t be placed on p r e v e n t i o n of q u e s t i o n a b l e data practices rather than on detection of p r o b l e m s after the fact, since there is no foolproof w a y to p r e v e n t falsification of data. It is worth noting that the p r o b l e m in the Boston clinic did not c o m e to light t h r o u g h any routine m o n i t o r i n g carried out by the study. The foremost d e f e n s e against such p r o b l e m s lies in the r e c r u i t m e n t and retention of honest, conscientious investigators w h o are selected for their scientific m a t u r i t y and integrity and w h o in turn are able to recruit s u p p o r t p e r s o n n e l w h o m a i n t a i n high s t a n d a r d s of integrity in the data collection process. At the s a m e time, s p o n s o r s of trials m u s t be p r e p a r e d to p r o v i d e a d e q u a t e s u p p o r t for essential activities, including those p e r f o r m e d b y the data center. U n d e r f u n d i n g , w i t h o u t a c o r r e s p o n d i n g cut back in the d e m a n d s placed on a center, can lead to serious p r o b l e m s in the trial. S p o n s o r s a n d leaders of trials m u s t be ,nindful of factors that m a y c o n t r i b u t e to p r o b l e m s in the s t u d y data and a t t e m p t to m i n i m i z e their influence. Peer p r e s s u r e from within the study, while a powerful incentive for i m p r o v i n g p e r f o r m a n c e , can result in faulty data if excessive. S p o n s o r s of trials a n d the policy b o a r d s that oversee them m u s t use restraint in issuing threats of f u n d i n g cutoffs as p r o d s for i m p r o v i n g p e r f o r m a n c e . Such threats m a y lead to shortcuts in the data collection process. Even if the principal i n v e s t i g a t o r of a clinic is i m p e r v i o u s to these threats, s u p p o r t p e r s o n n e l at the clinic w h o d r a w their salary from the s t u d y m a y not be. The contract m o d e of f u n d i n g , used in place of grant s u p p o r t in recent years by the National Institutes of Health to s u p p o r t various large scale multicenter trials, has e n h a n c e d the o p p o r t u n i t i e s for "fiscal h a r a s s m e n t " and for i m p l e m e n t i n g f u n d i n g cutoffs. In a d d i t i o n , the use of p a y m e n t s c h e d u l e s with the a m o u n t p r o v i d e d scaled to the n u m b e r of p a t i e n t s enrolled a n d followed, as u s e d in so,he i n d u s t r y - s p o n s o r e d trials, m a y lead to e n r o l l m e n t of u n a c c e p t a b l e patients and to the use of coercive t e c h n i q u e s to m a i n t a i n i n d i v i d u a l s u n d e r follow-up. P r o b l e m s c o n c e r n i n g the integrity of the data .nay arise from errors or p u r p o s e f u l data m a n i p u l a t i o n . Both k i n d s of p r o b l e m s m u s t be dealt with in a forthright m a n n e r to p r e s e r v e the integrity of the study. Major errors that c a n n o t be resolved after careful r e v i e w , especially those that are treatment-related, m a y require drastic action such as that used in the ECOG s t u d y w h e r e all data from the Boston clinic were p u r g e d from the central data files of the study. The H y p e r t e n s i o n Detection and Follow-up Program
Editorial
191 (HDFP) eliminated data from one of the clinics in that s t u d y because of a systematic error, which occurred d u r i n g the r a n d o m i z a t i o n process, that could not be resolved [14]. It m a y not be necessary to p u r g e faulty data if they are part of the information collected prior to r a n d o m i z a t i o n , since they cannot be treatment related in such cases. The Multiple Risk Factor Intervention Trial (MRFIT) elected to retain a small a m o u n t of discrepant data [15] from one of its clinics, since the discrepancies arose on examinations p e r f o r m e d before patients were assigned to treatment. A claim of data manipulation that arises after the study results have been p u b l i s h e d poses special problems. Responsible m e m b e r s of the scientific c o m m u n i t y should be wary of any u n s u b s t a n t i a t e d claim of data manipulation and should avoid assuming that absence of efforts by the investigators to a n s w e r the charge provides validity for the claim. If the claim is justified, new analyses should be carried out and reported after questionable data have been eliminated. This approach is not appropriate if the claim is without validity. Investigators from the study in this case must decide w h e t h e r or not to c o m m i t r e s o u r c e s to r e s p o n d to the claim. U G D P investigators, faced with such claims, elected to r e s p o n d to criticisms a p p e a r i n g in the scientific literature, but chose to ignore those appearing in the Medical Tribune and other u n r e v i e w e d periodicals. I n d e p e n d e n t audits of the s t u d y - - o n e by a committee a p p o i n t e d by the Biometrics Society and the other by the F D A - - w e r e instrumental in dispelling the claims in this case [16,17]. The j u d g e m e n t regarding the weight to be given to any set of published results should be based on assessment of the data quality. Published papers should p r o v i d e readers with a description of the p r o c e d u r e s used d u r i n g the s t u d y to insure the integrity of the data, including an account of any purge of faulty or e r r o n e o u s data. If m e t h o d s used for quality assurance and protection of data integrity are too detailed for inclusion in the paper containing the study results, they should be made available by publication of a separate m o n o g r a p h or through some other means. Several studies have d e v e l o p e d m o n o g r a p h s that provide information on the s t u d y design and baseline results leg, 8,18-20]. Reports of this kind could easily be e x p a n d e d to include the necessary information detailing data quality and integrity p r o c e d u r e s as well. The establishment of higher reporting standards for trials would ultimately provide a more realistic framework for evaluation of trials. As it stands at present, we are still in the paradoxical situation where a trial that pays little attention to data quality control and as a result has few if any data discrepancies to report, is v i e w e d as being superior to one with excellent o n g o i n g quality control procedures, but which, because of those p r o c e d u r e s has a n u m b e r of data deficiencies to report in the publications from the study. Controlled Clinical Trials encourages the s u b m i s s i o n of w o r t h w h i l e papers that deal with data quality and integrity procedures, w h e t h e r they relate to a specific trial or to a general class of trials. Such publications will provide a better u n d e r s t a n d i n g of the data problems in clinical trials and will help investigators to design and carry out better trials in the future. Curtis L. Meinert
192
Editorial
REFERENCES 1. Holden C: FDA tells senators of doctors who fake data in clinical drug trials. Science 206:432-433, 1979. 2. Boston Sunday Globe, June 29, 1980. 3. The Boston Globe, June 30, July 1-3, 1980. 4. Broad WJ: Would be academician pirates papers. Science 208:1438-1440, 1980. 5. Broad WJ: Imbroglio at Yale (I): Emergence of a fraud. Science 210:38-41, 1980. 6. Broad WJ: Imbroglio at Yale (II): A top job lost. Science 210:171-173, 1980. 7. Schey HM, Davis CE: The use of multiple measurements to monitor protocol adherence in epidemiological studies. Department of Biostatistics, University of North Carolina, Institute of Statistics, Mimeo Series, No. 1099, December, 1976. 8. Persantine-Aspirin Reinfarction Study Research Group: Design, methods and baseline results. Circulation 62 (part II), monograph, No. 71, 1980. 9. National Cooperative Gallstone Study Coordinating Center, Cedars-Sinai Medical Center, Los Angeles, California. 10. Kelsey FO: Biomedical monitoring. Journal of Clinical Pharmacology 18:3-9, 1978. 11. Department of Health, Education and Welfare, Food and Drug Administration: Clinical investigations: Proposed establishment of regulations on obligations of sponsors and monitors. Federal Register 42(Part IV):49612-49630, September 27, 1977. 12. Department of Health, Education, and Welfare, Food and Drug Administration: Obligations of clinical investigators of regulated articles: Prepared establishment of regulations. Federal Register 47(Part 47(Part I):35210-35236, August 8, 1978. 13. Doubts about Anturane. l.ancet 2:306-307, 1980. 14. Hypertension Detection and Follow-up Program Cooperative Group: Therapeutic control of blood pressure in the hypertension detection and follow-up program cooperative group. Prev Med 8:2-13, 1979. 15. St. Louis Globe Democrat, May 3, 1976. 16. Committee for the Assessment of Biometric Aspects of Controlled Trials of Hypoglycemic Agents: Report of the committee for the assessment of biometric aspects of controlled trials of hypoglycemic agents. JAMA 231:583-608, 1975. 17. Bilstad JM, Gurian JM, Lisook AB, Litt BD, Shanahaw EJ: The Food and Drug Administration Audit of the University Group Diabetes Program. Food and Drug Administration, October 16, 1978. 18. Klimt CR, Knatterud GI,, Meinert CL, Prout TE: The University Group Diabetes Program: A study of the effects of hypoglycemic agents on vascular complications in patients with adult-onset diabetes: I. Design methods and baseline results. Diabetes 19 (suppl 2):747-782, 1970. 19. The Coronary Drug Project Research Group: The coronary drug project: Design, methods and baseline results. Circulation 47 (suppl 1): 47:1-1 to 1-50, 1973. 20. Aspirin Myocardial Infarction Study Research Group: Aspirin myocardial infarction study: Design, methods and baseline results. U.S. Department of Health and Human Services, Public Health Service, National Institutes of Health, NIH Publication No. 80-2106, 198{).