- Email: [email protected]
Agreement on CDR ratings by committee
89S
Abstracts
An instrument has been developed from items considered to be relevant to the methodologic quality of a trial; that is, to minimise bias and increase precision. The relationship of the item to bias and precision needed to be supported by the scientific literature. A total of 69 potentially eligible items were considered of which 15 were included in the final instrument. The remaining items were considered to be important in evaluating other aspects of quality, including the trial's chances of success (the ability of the trial to answer the question being addressed, for example), or in describing the trial, but not relevant to bias or precision. The instrument has been piloted on 28 protocols submitted as part of an application tor financial support. A copy was completed for each protocol by a clinical expert, a statistician and an epidemiologist. Feedback provided by these reviewers was used to modify format, to re-word some questions and to formulate a set of instructions. It is now intended to assess the reliability and repeatability of the instrument, and to investigate ways of scoring each item and of combining individual scores into an overall quality score. P24 AGREEMENT ON CDR RATINGS BY COMMITTEE
Rochelle Tractenberg, Kimberly Schafer, Ron Thomas and John C. Morris
University of California, San Diego La Jolla, California The Clinical Dementia Rating (CDR) is a global measure of dementia representing five stages, ranging from "no impairment" (CDR 0) to "severe dementia" (CDR 3). The CDR is frequently used as an entry criterion and/or a primary outcome measure in Alzheimer's Disease clinical trials. The literature indicates three trends in CDR ratings; A) agreement among clinical raters is low on global CDR ratings of 0.5 and 1 relative to ratings of 0, 2, or 3; B) experience with the CDR does not improve agreement on these CDR ratings; and C) when raters disagree with the Gold Standard, they tend to overestimate the global CDR rating by at least one stage. We describe a method for evaluating concordance and consistency among multiple CDR raters. The Alzheimer's Disease Cooperative Study (ADCS) is planning a multi-center clinical trial of subjects with mild cognitive impairment (MCI). Individuals will be recruited for the study who have MCI with questionable dementia, a diagnosis partially supported by a global CDR 0.5. One possible outcome criterion is conversion from global CDR 0.5 to CDR 1. In the planned MCI trial, a committee of clinical raters will be confirming the CDR scores. To assure reliability and validity in the CDR ratings of the review committee, a certification procedure is proposed. Groups of "subjects" will be constructed using CDR worksheets and the CDR global rating algorithm. Worksheets, representing several permutations of each Global CDR stage, will be rated by each member of the committee. These ratings will be analyzed in terms of: 1 Consistency within the committee; 2. The chance-adjusted proportion of "subjects" correctly rated by each committee member; 3. Agreement of the committee with a Gold Standard; 4. The proportion of over/underestimations of the global CDRs for "subjects" at each level. Considerations of estimation, power, and precision will be discussed. We will also discuss how this approach can be used to establish the validity of oversight committee decisions t~r other clinical assessments used in multicenter clinical trials.
Abstracts
An instrument has been developed from items considered to be relevant to the methodologic quality of a trial; that is, to minimise bias and increase precision. The relationship of the item to bias and precision needed to be supported by the scientific literature. A total of 69 potentially eligible items were considered of which 15 were included in the final instrument. The remaining items were considered to be important in evaluating other aspects of quality, including the trial's chances of success (the ability of the trial to answer the question being addressed, for example), or in describing the trial, but not relevant to bias or precision. The instrument has been piloted on 28 protocols submitted as part of an application tor financial support. A copy was completed for each protocol by a clinical expert, a statistician and an epidemiologist. Feedback provided by these reviewers was used to modify format, to re-word some questions and to formulate a set of instructions. It is now intended to assess the reliability and repeatability of the instrument, and to investigate ways of scoring each item and of combining individual scores into an overall quality score. P24 AGREEMENT ON CDR RATINGS BY COMMITTEE
Rochelle Tractenberg, Kimberly Schafer, Ron Thomas and John C. Morris
University of California, San Diego La Jolla, California The Clinical Dementia Rating (CDR) is a global measure of dementia representing five stages, ranging from "no impairment" (CDR 0) to "severe dementia" (CDR 3). The CDR is frequently used as an entry criterion and/or a primary outcome measure in Alzheimer's Disease clinical trials. The literature indicates three trends in CDR ratings; A) agreement among clinical raters is low on global CDR ratings of 0.5 and 1 relative to ratings of 0, 2, or 3; B) experience with the CDR does not improve agreement on these CDR ratings; and C) when raters disagree with the Gold Standard, they tend to overestimate the global CDR rating by at least one stage. We describe a method for evaluating concordance and consistency among multiple CDR raters. The Alzheimer's Disease Cooperative Study (ADCS) is planning a multi-center clinical trial of subjects with mild cognitive impairment (MCI). Individuals will be recruited for the study who have MCI with questionable dementia, a diagnosis partially supported by a global CDR 0.5. One possible outcome criterion is conversion from global CDR 0.5 to CDR 1. In the planned MCI trial, a committee of clinical raters will be confirming the CDR scores. To assure reliability and validity in the CDR ratings of the review committee, a certification procedure is proposed. Groups of "subjects" will be constructed using CDR worksheets and the CDR global rating algorithm. Worksheets, representing several permutations of each Global CDR stage, will be rated by each member of the committee. These ratings will be analyzed in terms of: 1 Consistency within the committee; 2. The chance-adjusted proportion of "subjects" correctly rated by each committee member; 3. Agreement of the committee with a Gold Standard; 4. The proportion of over/underestimations of the global CDRs for "subjects" at each level. Considerations of estimation, power, and precision will be discussed. We will also discuss how this approach can be used to establish the validity of oversight committee decisions t~r other clinical assessments used in multicenter clinical trials.