- Email: [email protected]
Gallbladder disease—I. Assessment of validity and reliability of data derived from a questionnaire
J ChronDis 1976. Vol. 29, pp. 51-58. Pergamon Press. Printed in Great Britain
GALLBLADDER DISEASE-I. ASSESSMENT OF VALIDITY AND RELIABILITY OF DATA DERIVED FROM A QUESTIONNAIRE A STUDY OF 62,739 WEIGHT-CONSCIOUS
WOMEN
RONALD A. BERNSTEIN,ELDRED E. GIEFERand ALFRED A. RIMM Department
of Preventive Medicine, Milwaukee, WI 53223
(Received in revised form 22 December
1974)
THOUGH the well used Cornell Medical Index questionnaire is of little value as an indicator of the occurrence of specific disorders [I, 23, others have found their questionnaires are effective for deriving prevalence rates of certain morbidity conditions [3, 43. In certain epidemiological investigations where clinical examination of all participants is not possible, and where prevalence of morbidity is of interest, the investigator must resort to a questionnaire. It has been demonstrated that various factors such as the wording of the questions, age, sex and ethnic background [2, 3, 5j are associated with incorrect responses on the questionnaire. The purpose of this study is to determine the validity of the response to questions concerning the presence or absence of gallbladder disease in a large female population. The results of this study suggest that the responses of women answering the gallbladder question are in good agreement with their medical records and, therefore, are useful for estimating prevalence in different segments of the population under study
161. METHODS
In 1969 the TOPS (Take off Pounds Sensibly) Club, Inc., with approximately 200,000 members, initiated an investigation using a questionnaire to obtain a better understanding of the etiology and consequences of obesity. All members of TOPS have weight control problems and recognized that information about themselves could contribute to a better understanding of their common problem. Nearly 125,000 questionnaires were mailed in February, March and April 1969 [7]. More than 40,000 questionnaires were returned blank, mainly because of the absence of members at the weekly club meeting. A total of 83,980 questionnaires was returned and reviewed for inconsistencies. During the 3 yr after the collection of the data it was coded, punched, edited and computerized. A data retrieval program called SCALPELwas written [8]. About 10,000 persons did not complete Part II of the questionnaire. Therefore, there were 73,532 persons with complete information as previousIy reported [9]. Women less than 20 yr of age, diabetic and/or pregnant women were excluded from 51
52
RONALD
A. BERNSTEIN,ELDREDE. GIEFERand ALFREDA. RIMM
White females
FIG. 1. Derivation
of TOPS population
under study.
this study. There were 52,884 women reporting no history of GBD and 9855 women with a history of GBD for a total of 62,739. The derivation of the TOPS population under study is shown in Fig. 1. Self-administered questionnaires have been employed in many similar studies [lO-121. This technique of data gathering has the advantage of uniformity of administration and, in addition, is an economical way to obtain a large amount of historical information. There were three unique aspects of the population under study: 1. The women who participated volunteered to answer the two-part questionnaire. In addition, they donated money to support this research. Their common problems of weight control promoted an esprit de corps which was reflected in the thoroughness of their answers. 2. The first part of the questionnaire was administered to the women in small groups averaging 20 per group at the weekly TOPS meeting. The leaders of the groups were given step-by-step instructions for completion of the first part of the questionnaire. In pilot tests of this method of administration it took the women approximately 90 min to complete Part I. Part II of the questionnaire included written instructions and was answered at home. It contained questions which required recall of detail and reference to family records for such events as births and deaths. The questionnaires were returned to the group leader at the following weekly meeting in a sealed envelope. 3. The women who belong to TOPS are instructed, as part of the program for weight control, to deal honestly and directly with their health problems. An important rule of the organization is for each member to consult with her private physician for management of her weight problem.
The women who participated in this program were highly motivated and under the direct care of their physician. This would seem to be a unique situation; consequently, the estimates of validity of questionnaire data from this population are probably not representative of those from a random sample of women.
Gallbladder
Disease-I
53
TOPS members questionnaire The exact question used for determining the history of gallbladder disease was ‘Has a doctor ever told you that you had gallbladder disease?‘. This type of question has been used previously for determining history of morbidity conditions [13]. If the answer to the question was ‘Yes’ the women were asked to record diagnosis, duration of symptoms, treatment and age of onset. One question on the questionnaire asked for the name and address of their physician. Authorization was also requested for release of information from their physician. Physicians questionnaire A two page questionnaire was sent to physicians to obtain clinical information. A total of 20,000 physician questionnaires were mailed. Questions pertained to morbidity conditions and psychological stability of the patient. Only about 42% of the physicians answered and returned the questionnaire. In a preliminary analysis to determine whether the responding physicians were representative of all physicians in the study, nearly 10,000 physicians’ questionnaires were tabulated and compared to the original answers by the corresponding TOPS member. It was found that the rate of gallbladder disease prevalence reported by the women was much higher than that reported by their physicians. To further evaluate the validity of the physician questionnaire, a follow-up study was conducted which consisted of telephone interviews of physicians. The physicians who were called were a random sample of those who marked on the physician questionnaire that their patient had no history of gallbladder disease although she indicated a history of the disease on the questionnaire. The first 10 physicians interviewed indicated that their report stating no gallbladder disease was incorrect. Interviewing was discontinued since it was apparent that the physician questionnaire contained many false negative results. There are several possible explanations for the poor validity of a questionnaire mailed to physicians : I. The patient may have changed physicians and the particular condition was treated by a previous physician. 2. The physician may not have thoroughly completed the questionnaire. 3. The physician may not have thoroughly searched the medical records of the patient.
To determine whether the physicians response was poorer when the onset of disease was in the more distant past, the data were summarized as shown in Table 1. There were 6328 who reported a history of GBD and gave age of onset. Of these there were 740 whose physician returned his questionnaire. There were 590physicians who reported no history with a mean interval of 10.4 yr between onset and 1969. This was significantly higher than the mean of 6.4 yr for the physicians who reported a history of disease. As a result of the poor validity of the physician’s questionnaire, it was determined that telephone interviews of the physician would be the preferred basis for estimating the validity of the gallbladder disease responses by the TOPS members. VALIDITY
ESTIMATES
A study of validity of the member’s questionnaires focused on a sample of 100 women who answered ‘Yes’ to the gallbladder question, Fifty women were chosen at random from each of two groups.
RONALDA.BERNSTJW,ELDRED
54
E. Grassland
ALFRED A.&MM
TABLE 1. ROLE OF TIME FROM DIAGNOSIS' IN DETERMININGVALIDITYOF GBD QUESTIONNAIRE
RESPONSEFROM
Physician's Response to GBD Question YES
NO
150
590
Number ............... Mean2 .............. S.D.
..............
6.353
NOT AVAILABLE 5588
10.39
9.40
7.94
7.93
6.31
TOTAL =
6328
1. Measuredby looking at (Present agehsrp-Age at onset. 2. Only women who answered GBD ‘Yes’ with age onset information. 3. Significantly different Yes from No (p < 0.0005). Group I. Those who substantiated as date of surgery.
their ‘Yes’ answer with additional narrative information,
such
Group II. Those who gave a ‘Yes’ answer but did not add any additional information.
The physicians of these members were interviewed by telephone. The physician was asked whether his patient ever had gallbladder disease. He was asked about the diagnosis, impression or symptoms that he considered as noted in his records, or whether there was some record from another physician or the patient herself, which could be possibly related to gallbladder disease prior to November, 1969. If this were positive, further documentation was sought consisting of information from pathological reports, medications, X-ray findings, etc. The categories of confirmation of disease by the physician were grouped as follows : 1. Definitely confirmed and documented by report of pathological specimens and/or positive X-ray findings. 2. Confirmed diagnosis of disease by the physician without objective findings in the medical record. 3. Confirmed by medical records history without further documentation. 4. Unconfirmed
because of lack of information.
5. Positive denial of history of gallbladder disease.
Categories l-3 were used to obtain one estimate of validity of the TOPS members questionnaire. Responses in Category 4 were eliminated from further calculations. This classification scheme was similar to that employed in the Framingham Study [14]. Estimates of the validity of the questionnaire in each group using over-reporting rates were obtained as follows: Per cent overreporting =lOO-
Number of physicians giving a positive diagnosis to GBD x 100 Total number physicians interviewed ( >*
If for example, 45 physicians of women in Group I confirmed the patients’ positive response (Category l-3) to the gallbladder question, then there would be 10% overreporting by the TOPS members. RELIABILITY
ESTIMATES
To estimate the reliability of the gallbladder question on the members questionnaire, a random sample of 100 women who answered ‘Yes’ were selected and interviewed by
55
Gallbladder Disease-I
telephone. Answers from the questionnaire were compared to those from the telephone interview. Since the interval between completion of the questionnaire and the telephone interview was between 3 and 4 yr the women were asked about their history of gallbladder disease prior to 1969. RESULTS
There were 62,739 completed questionnaires included in this study. Of the 9855 women who responded positively to a history of gallbladder disease, 64% (6328) included a complete narrative on their disease including their age at onset (Group I). This data was used for the computation of life tables for gallbladder disease [15]. Validity: Members andphysicians answers compared
For the 50 women in Group I who reported gallbladder disease and whose date of onset was recorded, it was found that 49 cases of gallbladder disease (98%) were confirmed by documenting evidence from the physician’s interview. Among the 50 women whose date of onset was not recorded (Group II), 68% of the diagnosis could be definitely documented (Table 2). TABLE
2.
ESTIMATE OF OVBRREPORTING OF GALLBLADDER DISEASE: PH~~CIAN TO TOPS MEMBERQUESTIONNAIRE
GROUP
NUMBER OF PHYSICIANS WHO CONFIRMED "YES"
NUMBER OF WOMEN WHO ANSWERED "YES"
INT~~RVIEW COMPARED
PERCENT OVERREPORTING
I
50
49
2%
II
50
34
32%
It is assumed the data from the physicians who reported gallbladder disease even though the women did not give a history of disease is reliable and can be used for the determination of underreporting. An estimate of patient underreporting was obtained by comparing the woman’s response (viz.-‘No’ gallbladder disease) with her physician’s response. Utilizing 9832 physicians questionnaires, it was found that there were 157 patients who reported a negative history of gallbladder where the physician questionnaire reflected a positive history of gallbladder disease. Reliability of members questionnaires
Of the total group of members who completed questionnaires and were telephoned (IOO), 87% gave the same response to the gallbladder disease question. Narrative answers, including diagnosis, year of diagnosis, treatment and duration of symptoms were closely matched for these women. DISCUSSION
Utilizing the estimates of underreporting and overreporting it is now possible to obtain an adjusted estimated prevalence of gallbladder disease for these women. There were 9855 women of the 11,175 who gave a ‘Yes’ answer to the gallbladder disease question who were not diabetic, not pregnant and not under 20 yr of age. A total of 6328 women gave dates of disease and 3527 did not give any dates. There were 52,884
56
RONALD A. BERNSTEIN,ELDRED E. GIEFERand ALFRED A. RIMM
women who reported no history of gallbladder disease. Using only responses from member questionnaires, the calculated (crude) prevalence rate for gallbladder disease was 15.7 per 100 women. Utilizing results from the telephone interview with patient’s doctors and separate doctor questionnaires, estimates of the number of patients giving incorrect answers were obtained (Table 3). The methods of estimation are given in the appendix. TABLE 3.
ESTIMATED
NUMBER OF TOPS WOMEN RESPONSESWHICH AGREED AND DISAGREEDWITH CONDITIONOF GALLBLADDER
TOPS MRMBER RESPONSE (from questionnaire) YES NO
TOTAL
CONDITION OF GALLBLADDERI TOTAL NOT DISEASED DISEASED 8599
1256
9655
846
52,038
52,884
9445
53,294
62,739
1. The method for the estimation of this data is described in the appendix.
When correcting for under- and overreporting, new prevalence is calculated to be:
using the data from Table 3, the
Corrected Prevalence= 85;;;346=0.1506 ,
with a standard error of 0.0014 [16]. It is interesting that this adjusted rate of 15.1 per 100 women is similar to the crude rate of 15.7. In consideration of the reliability and validity of data derived from questionnaires, Madow [13] has reached the following conclusions: 1. Respondents report more fully on conditions important to them; unreported conditions tend to be those where there was little discomfort and/or anxiety. 2. Diseases of the gallbladder are conditions with low indices of over- and underreporting. 3. In questionnaire studies, there is a tendency for over- and underreporting to balance out and for the net bias to be relatively small.
How well does the present estimate of prevalence compare with other studies? The adjusted prevalence of gallbladder disease (15.1) in nondiabetic TOPS women over 20 yr of age is considerably higher than the 5.8 per 100 found in the Framingham Study [14]. The difference between the two studies is probably attributable to the difference in populations studied. The TOPS population includes women who are presently overweight or who have been overweight. Since obesity is an apparent risk factor in gallbladder disease 163it is not surprising to find a higher prevalence in this population. There is no evidence that the higher rate in this study is due to gross overreporting of gallbladder disease. Overreporting was estimated from two random samples of all women answering ‘Yes’ to gallbladder disease on the questionnaire. Women in Group I constituted 64% of total ‘Yes’. Therefore, the estimate of overreporting was weighted for this as follows :
Gallbladder
Group I Group II
57
Disease-I
o/oOverreporting 0.64 x 02=01.28 0.36 x 32= 11.45 12.73
Thus, it is estimated that 12.7% of all women giving a history of gallbladder disease, in fact, did not have a history of disease. Results of over 52,000 TOPS members who reported no history of gallbladder disease show less than 1.6% underreporting based upon 9832 doctors questionnaires. We feel, therefore, that the incidence of underreporting of gallbladder disease, a disease which has relatively large impact on the patient, is minimal. Because of the unreliability of the doctors questionnaire one assumes this to be a conservative estimate of underreporting. However, even if this number were severat times greater, the resultant net bias would be further reduced and the adjusted occurrence rate would be still closer to the crude rate. Several physicians, when telephoned, offered a diagnosis other than gallbladder disease although symptoms were similar to that of gallbfadder disease. This could have caused confusion on the patients’ part concerning diagnosis and thus, account for some of the overreporting. One other problem encountered was the fact that many physicians had only seen the patient for a few years, especiahy during the period that the questionnaires were being completed. In a few cases, the doctor was only a consultant and did not have access to the entire medical records. It was quite evident that many women had their disease treated elsewhere by physicians differing from those named in the questionnaire. It was found that 65% of the women who were questioned by telephone gave a different name for the physician who treated their gallbladder disease than the physician named in the questionnaire, their personal physician. Other studies on gallbladder disease [14, 171are considered ‘better’ than the type of study reported here because verification of disease was from hospital records. While these studies can be considerably more difficult and time consuming to carry out, their estimate of prevalence of disease is perhaps no better than that of the present questionnaire study. SUMMARY
This study included 62,739 women who belong to TOPS (Take Off Pounds Sensibly), and this is perhaps the largest study of gallbladder disease in women ever conducted. The main findings of this study are: 1. The prevalence of gallbladder disease, adjusted for over- and underreporting, was calculated to be 15.1 per 100 women and was not appreciably different from the unadjusted rate of 15.7 previously reported [6]. 2. The estimated reliability of the gallbladder question was found to lx 87 %. 3. When a small sample of doctors whose questionnaire indicated absence of gallbladder disease and therefore, disagreed with the patient’s history of gallbladder disease were requestioned, they changed their response. 4. There was 98 % physician confirmation (telephone interviews) of positive diagnosis of gallbladder disease for women who completed the narrative section of the question; 68% of the positive diagnosis of gallbladder disease for women without the narrative section of the questionnaire could be confirmed by the physician. 5. Underreporting of gallbladder disease was conservatively estimated to be 1.6% of the total number with disease. 6. Overreporting of gallbladder disease was estimated to be 12.7% of the total number with disease.
58
RONALDA. BERNSTEIN,ELDREDE. GIEPERand ALFREDA. R~MM
Acknowledgement-This work is supported by TOPS Club, Inc., Obesity and Metabolic Program of Deaconess Hospital, Milwaukee, Wisconsin.
Research
REFERENCES 1.
2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17.
Abramson JH, Terespolsky L, Brook JG, et al.: Cornell Medical Index as a health measure in epidemiological studies: A test of the validity of a health questionnaire. Brit J Prev Sot Med 19: 103-110, 1965 Abramson JH: The Cornell Medical Index as an epidemiological tool. AJPH 56: 287-298, 1966 Collen MF, Catler JL, Siegelaub AB, et al.: Reliability of a self-administered medical questionnaire. Arch Int Med 123: 664681, 1969 Alexion NG, Wiener G: Reliability of a self-administered health questionnaire for secondary school students. Am J Pub1 Hlth 58: 1439-1446, 1968 Epstein LM: Validity of a questionnaire for diagnosis of peptic ulcer in an ethnically heterogeneous population. J Chron Dis 22: 49-57, 1970 Bernstein RA, Werner LH, Rimm AA: Relationship of gallbladder disease to parity, obesity and age: A study of 62,739 weight-conscious women. Hlth Sew Rep 88: 925-936, 1973 Rimm I, Rimm A: TOPS metabolic and obesity research: International research questionnaire. Prev Med 3 : 543-572, 1974 Kay H, et al.: SCALPEL:prototype for a computer assisted finite language for data screening. table method in obtaining clinically useful information. J Chron Dis: to be published Rimm AA, Werner LH, Bernstein RA: Disease and obesity in 73,532 women. Obesity/Bariatric Med 1: 77-84, 1972 Hershberg PI: Medical diagnosis: The role of a brief open-ended medical history questionnaire. J Med Educ 44: 293-297, 1969 Brodman K, Erdmann AJ, Jr, Lorge I, et al.: Cornell Medical Index-health questionnaire: As diagnostic instrument. JAMA 145: 152-157, 1951 Hutchison GB, Shapiro S, Densen PM: Evaluation of a mailed questionnaire. Am J Pub Hlth 52: 1894-1917, 1962 Madow W: Net differences in interview data on chronic conditions and information derived from medical records. Vital Health Stat, Series 2, 57: 1-9, 1973 Friedman GO, Kannel WB, Danter TR: The epidemiology of gallbladder disease: Observation in the Framingham Study. J Chron Dis 19: 273-292, 1966 Bernstein RA, Giefer EE, Werner LH, et al: Gallbladder disease-II. Utilization the of life table method.in obtaining clinically useful information. J Chron Dis: to be published Fleiss JL: Statistical Methods for Rates and Proportions, pp. 141-143. New York: John Wiley 1973 Nelson BD, Powaznik J, Benfield JR: Gallbladder disease in southwestern American Indians. Arch Surg 103: 41-43, 1971 APPENDIX
Procedurefor the calculation of estimates for Table 3 To estimate the number of women who incorrectly reported gallbladder sum of estimates from the study of the two groups of 50 patients each: Group I: 0.02 x 6328= 127 Group II: 0.32 x 3527= 1129
disease, we can use the
1256. The estimate of women who correctly reported gallbladder
disease is, thus, calculated to be:
(6328-127)+(3527-1129)=8599. If we assume a rate of 1.6% underreporting, which is probably conservative since it has been shown above that the physician questionnaire was relatively invalid, an estimate for the total number of women who underreported can be calculated as follows: 0.016x (62,739-9855)=846. Similarly the last value in the table, the estimates for number of women answering gallbladder disease ‘No’ correctly can be calculated to be: 52,884-846=52,038.
GALLBLADDER DISEASE-I. ASSESSMENT OF VALIDITY AND RELIABILITY OF DATA DERIVED FROM A QUESTIONNAIRE A STUDY OF 62,739 WEIGHT-CONSCIOUS
WOMEN
RONALD A. BERNSTEIN,ELDRED E. GIEFERand ALFRED A. RIMM Department
of Preventive Medicine, Milwaukee, WI 53223
(Received in revised form 22 December
1974)
THOUGH the well used Cornell Medical Index questionnaire is of little value as an indicator of the occurrence of specific disorders [I, 23, others have found their questionnaires are effective for deriving prevalence rates of certain morbidity conditions [3, 43. In certain epidemiological investigations where clinical examination of all participants is not possible, and where prevalence of morbidity is of interest, the investigator must resort to a questionnaire. It has been demonstrated that various factors such as the wording of the questions, age, sex and ethnic background [2, 3, 5j are associated with incorrect responses on the questionnaire. The purpose of this study is to determine the validity of the response to questions concerning the presence or absence of gallbladder disease in a large female population. The results of this study suggest that the responses of women answering the gallbladder question are in good agreement with their medical records and, therefore, are useful for estimating prevalence in different segments of the population under study
161. METHODS
In 1969 the TOPS (Take off Pounds Sensibly) Club, Inc., with approximately 200,000 members, initiated an investigation using a questionnaire to obtain a better understanding of the etiology and consequences of obesity. All members of TOPS have weight control problems and recognized that information about themselves could contribute to a better understanding of their common problem. Nearly 125,000 questionnaires were mailed in February, March and April 1969 [7]. More than 40,000 questionnaires were returned blank, mainly because of the absence of members at the weekly club meeting. A total of 83,980 questionnaires was returned and reviewed for inconsistencies. During the 3 yr after the collection of the data it was coded, punched, edited and computerized. A data retrieval program called SCALPELwas written [8]. About 10,000 persons did not complete Part II of the questionnaire. Therefore, there were 73,532 persons with complete information as previousIy reported [9]. Women less than 20 yr of age, diabetic and/or pregnant women were excluded from 51
52
RONALD
A. BERNSTEIN,ELDREDE. GIEFERand ALFREDA. RIMM
White females
FIG. 1. Derivation
of TOPS population
under study.
this study. There were 52,884 women reporting no history of GBD and 9855 women with a history of GBD for a total of 62,739. The derivation of the TOPS population under study is shown in Fig. 1. Self-administered questionnaires have been employed in many similar studies [lO-121. This technique of data gathering has the advantage of uniformity of administration and, in addition, is an economical way to obtain a large amount of historical information. There were three unique aspects of the population under study: 1. The women who participated volunteered to answer the two-part questionnaire. In addition, they donated money to support this research. Their common problems of weight control promoted an esprit de corps which was reflected in the thoroughness of their answers. 2. The first part of the questionnaire was administered to the women in small groups averaging 20 per group at the weekly TOPS meeting. The leaders of the groups were given step-by-step instructions for completion of the first part of the questionnaire. In pilot tests of this method of administration it took the women approximately 90 min to complete Part I. Part II of the questionnaire included written instructions and was answered at home. It contained questions which required recall of detail and reference to family records for such events as births and deaths. The questionnaires were returned to the group leader at the following weekly meeting in a sealed envelope. 3. The women who belong to TOPS are instructed, as part of the program for weight control, to deal honestly and directly with their health problems. An important rule of the organization is for each member to consult with her private physician for management of her weight problem.
The women who participated in this program were highly motivated and under the direct care of their physician. This would seem to be a unique situation; consequently, the estimates of validity of questionnaire data from this population are probably not representative of those from a random sample of women.
Gallbladder
Disease-I
53
TOPS members questionnaire The exact question used for determining the history of gallbladder disease was ‘Has a doctor ever told you that you had gallbladder disease?‘. This type of question has been used previously for determining history of morbidity conditions [13]. If the answer to the question was ‘Yes’ the women were asked to record diagnosis, duration of symptoms, treatment and age of onset. One question on the questionnaire asked for the name and address of their physician. Authorization was also requested for release of information from their physician. Physicians questionnaire A two page questionnaire was sent to physicians to obtain clinical information. A total of 20,000 physician questionnaires were mailed. Questions pertained to morbidity conditions and psychological stability of the patient. Only about 42% of the physicians answered and returned the questionnaire. In a preliminary analysis to determine whether the responding physicians were representative of all physicians in the study, nearly 10,000 physicians’ questionnaires were tabulated and compared to the original answers by the corresponding TOPS member. It was found that the rate of gallbladder disease prevalence reported by the women was much higher than that reported by their physicians. To further evaluate the validity of the physician questionnaire, a follow-up study was conducted which consisted of telephone interviews of physicians. The physicians who were called were a random sample of those who marked on the physician questionnaire that their patient had no history of gallbladder disease although she indicated a history of the disease on the questionnaire. The first 10 physicians interviewed indicated that their report stating no gallbladder disease was incorrect. Interviewing was discontinued since it was apparent that the physician questionnaire contained many false negative results. There are several possible explanations for the poor validity of a questionnaire mailed to physicians : I. The patient may have changed physicians and the particular condition was treated by a previous physician. 2. The physician may not have thoroughly completed the questionnaire. 3. The physician may not have thoroughly searched the medical records of the patient.
To determine whether the physicians response was poorer when the onset of disease was in the more distant past, the data were summarized as shown in Table 1. There were 6328 who reported a history of GBD and gave age of onset. Of these there were 740 whose physician returned his questionnaire. There were 590physicians who reported no history with a mean interval of 10.4 yr between onset and 1969. This was significantly higher than the mean of 6.4 yr for the physicians who reported a history of disease. As a result of the poor validity of the physician’s questionnaire, it was determined that telephone interviews of the physician would be the preferred basis for estimating the validity of the gallbladder disease responses by the TOPS members. VALIDITY
ESTIMATES
A study of validity of the member’s questionnaires focused on a sample of 100 women who answered ‘Yes’ to the gallbladder question, Fifty women were chosen at random from each of two groups.
RONALDA.BERNSTJW,ELDRED
54
E. Grassland
ALFRED A.&MM
TABLE 1. ROLE OF TIME FROM DIAGNOSIS' IN DETERMININGVALIDITYOF GBD QUESTIONNAIRE
RESPONSEFROM
Physician's Response to GBD Question YES
NO
150
590
Number ............... Mean2 .............. S.D.
..............
6.353
NOT AVAILABLE 5588
10.39
9.40
7.94
7.93
6.31
TOTAL =
6328
1. Measuredby looking at (Present agehsrp-Age at onset. 2. Only women who answered GBD ‘Yes’ with age onset information. 3. Significantly different Yes from No (p < 0.0005). Group I. Those who substantiated as date of surgery.
their ‘Yes’ answer with additional narrative information,
such
Group II. Those who gave a ‘Yes’ answer but did not add any additional information.
The physicians of these members were interviewed by telephone. The physician was asked whether his patient ever had gallbladder disease. He was asked about the diagnosis, impression or symptoms that he considered as noted in his records, or whether there was some record from another physician or the patient herself, which could be possibly related to gallbladder disease prior to November, 1969. If this were positive, further documentation was sought consisting of information from pathological reports, medications, X-ray findings, etc. The categories of confirmation of disease by the physician were grouped as follows : 1. Definitely confirmed and documented by report of pathological specimens and/or positive X-ray findings. 2. Confirmed diagnosis of disease by the physician without objective findings in the medical record. 3. Confirmed by medical records history without further documentation. 4. Unconfirmed
because of lack of information.
5. Positive denial of history of gallbladder disease.
Categories l-3 were used to obtain one estimate of validity of the TOPS members questionnaire. Responses in Category 4 were eliminated from further calculations. This classification scheme was similar to that employed in the Framingham Study [14]. Estimates of the validity of the questionnaire in each group using over-reporting rates were obtained as follows: Per cent overreporting =lOO-
Number of physicians giving a positive diagnosis to GBD x 100 Total number physicians interviewed ( >*
If for example, 45 physicians of women in Group I confirmed the patients’ positive response (Category l-3) to the gallbladder question, then there would be 10% overreporting by the TOPS members. RELIABILITY
ESTIMATES
To estimate the reliability of the gallbladder question on the members questionnaire, a random sample of 100 women who answered ‘Yes’ were selected and interviewed by
55
Gallbladder Disease-I
telephone. Answers from the questionnaire were compared to those from the telephone interview. Since the interval between completion of the questionnaire and the telephone interview was between 3 and 4 yr the women were asked about their history of gallbladder disease prior to 1969. RESULTS
There were 62,739 completed questionnaires included in this study. Of the 9855 women who responded positively to a history of gallbladder disease, 64% (6328) included a complete narrative on their disease including their age at onset (Group I). This data was used for the computation of life tables for gallbladder disease [15]. Validity: Members andphysicians answers compared
For the 50 women in Group I who reported gallbladder disease and whose date of onset was recorded, it was found that 49 cases of gallbladder disease (98%) were confirmed by documenting evidence from the physician’s interview. Among the 50 women whose date of onset was not recorded (Group II), 68% of the diagnosis could be definitely documented (Table 2). TABLE
2.
ESTIMATE OF OVBRREPORTING OF GALLBLADDER DISEASE: PH~~CIAN TO TOPS MEMBERQUESTIONNAIRE
GROUP
NUMBER OF PHYSICIANS WHO CONFIRMED "YES"
NUMBER OF WOMEN WHO ANSWERED "YES"
INT~~RVIEW COMPARED
PERCENT OVERREPORTING
I
50
49
2%
II
50
34
32%
It is assumed the data from the physicians who reported gallbladder disease even though the women did not give a history of disease is reliable and can be used for the determination of underreporting. An estimate of patient underreporting was obtained by comparing the woman’s response (viz.-‘No’ gallbladder disease) with her physician’s response. Utilizing 9832 physicians questionnaires, it was found that there were 157 patients who reported a negative history of gallbladder where the physician questionnaire reflected a positive history of gallbladder disease. Reliability of members questionnaires
Of the total group of members who completed questionnaires and were telephoned (IOO), 87% gave the same response to the gallbladder disease question. Narrative answers, including diagnosis, year of diagnosis, treatment and duration of symptoms were closely matched for these women. DISCUSSION
Utilizing the estimates of underreporting and overreporting it is now possible to obtain an adjusted estimated prevalence of gallbladder disease for these women. There were 9855 women of the 11,175 who gave a ‘Yes’ answer to the gallbladder disease question who were not diabetic, not pregnant and not under 20 yr of age. A total of 6328 women gave dates of disease and 3527 did not give any dates. There were 52,884
56
RONALD A. BERNSTEIN,ELDRED E. GIEFERand ALFRED A. RIMM
women who reported no history of gallbladder disease. Using only responses from member questionnaires, the calculated (crude) prevalence rate for gallbladder disease was 15.7 per 100 women. Utilizing results from the telephone interview with patient’s doctors and separate doctor questionnaires, estimates of the number of patients giving incorrect answers were obtained (Table 3). The methods of estimation are given in the appendix. TABLE 3.
ESTIMATED
NUMBER OF TOPS WOMEN RESPONSESWHICH AGREED AND DISAGREEDWITH CONDITIONOF GALLBLADDER
TOPS MRMBER RESPONSE (from questionnaire) YES NO
TOTAL
CONDITION OF GALLBLADDERI TOTAL NOT DISEASED DISEASED 8599
1256
9655
846
52,038
52,884
9445
53,294
62,739
1. The method for the estimation of this data is described in the appendix.
When correcting for under- and overreporting, new prevalence is calculated to be:
using the data from Table 3, the
Corrected Prevalence= 85;;;346=0.1506 ,
with a standard error of 0.0014 [16]. It is interesting that this adjusted rate of 15.1 per 100 women is similar to the crude rate of 15.7. In consideration of the reliability and validity of data derived from questionnaires, Madow [13] has reached the following conclusions: 1. Respondents report more fully on conditions important to them; unreported conditions tend to be those where there was little discomfort and/or anxiety. 2. Diseases of the gallbladder are conditions with low indices of over- and underreporting. 3. In questionnaire studies, there is a tendency for over- and underreporting to balance out and for the net bias to be relatively small.
How well does the present estimate of prevalence compare with other studies? The adjusted prevalence of gallbladder disease (15.1) in nondiabetic TOPS women over 20 yr of age is considerably higher than the 5.8 per 100 found in the Framingham Study [14]. The difference between the two studies is probably attributable to the difference in populations studied. The TOPS population includes women who are presently overweight or who have been overweight. Since obesity is an apparent risk factor in gallbladder disease 163it is not surprising to find a higher prevalence in this population. There is no evidence that the higher rate in this study is due to gross overreporting of gallbladder disease. Overreporting was estimated from two random samples of all women answering ‘Yes’ to gallbladder disease on the questionnaire. Women in Group I constituted 64% of total ‘Yes’. Therefore, the estimate of overreporting was weighted for this as follows :
Gallbladder
Group I Group II
57
Disease-I
o/oOverreporting 0.64 x 02=01.28 0.36 x 32= 11.45 12.73
Thus, it is estimated that 12.7% of all women giving a history of gallbladder disease, in fact, did not have a history of disease. Results of over 52,000 TOPS members who reported no history of gallbladder disease show less than 1.6% underreporting based upon 9832 doctors questionnaires. We feel, therefore, that the incidence of underreporting of gallbladder disease, a disease which has relatively large impact on the patient, is minimal. Because of the unreliability of the doctors questionnaire one assumes this to be a conservative estimate of underreporting. However, even if this number were severat times greater, the resultant net bias would be further reduced and the adjusted occurrence rate would be still closer to the crude rate. Several physicians, when telephoned, offered a diagnosis other than gallbladder disease although symptoms were similar to that of gallbfadder disease. This could have caused confusion on the patients’ part concerning diagnosis and thus, account for some of the overreporting. One other problem encountered was the fact that many physicians had only seen the patient for a few years, especiahy during the period that the questionnaires were being completed. In a few cases, the doctor was only a consultant and did not have access to the entire medical records. It was quite evident that many women had their disease treated elsewhere by physicians differing from those named in the questionnaire. It was found that 65% of the women who were questioned by telephone gave a different name for the physician who treated their gallbladder disease than the physician named in the questionnaire, their personal physician. Other studies on gallbladder disease [14, 171are considered ‘better’ than the type of study reported here because verification of disease was from hospital records. While these studies can be considerably more difficult and time consuming to carry out, their estimate of prevalence of disease is perhaps no better than that of the present questionnaire study. SUMMARY
This study included 62,739 women who belong to TOPS (Take Off Pounds Sensibly), and this is perhaps the largest study of gallbladder disease in women ever conducted. The main findings of this study are: 1. The prevalence of gallbladder disease, adjusted for over- and underreporting, was calculated to be 15.1 per 100 women and was not appreciably different from the unadjusted rate of 15.7 previously reported [6]. 2. The estimated reliability of the gallbladder question was found to lx 87 %. 3. When a small sample of doctors whose questionnaire indicated absence of gallbladder disease and therefore, disagreed with the patient’s history of gallbladder disease were requestioned, they changed their response. 4. There was 98 % physician confirmation (telephone interviews) of positive diagnosis of gallbladder disease for women who completed the narrative section of the question; 68% of the positive diagnosis of gallbladder disease for women without the narrative section of the questionnaire could be confirmed by the physician. 5. Underreporting of gallbladder disease was conservatively estimated to be 1.6% of the total number with disease. 6. Overreporting of gallbladder disease was estimated to be 12.7% of the total number with disease.
58
RONALDA. BERNSTEIN,ELDREDE. GIEPERand ALFREDA. R~MM
Acknowledgement-This work is supported by TOPS Club, Inc., Obesity and Metabolic Program of Deaconess Hospital, Milwaukee, Wisconsin.
Research
REFERENCES 1.
2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17.
Abramson JH, Terespolsky L, Brook JG, et al.: Cornell Medical Index as a health measure in epidemiological studies: A test of the validity of a health questionnaire. Brit J Prev Sot Med 19: 103-110, 1965 Abramson JH: The Cornell Medical Index as an epidemiological tool. AJPH 56: 287-298, 1966 Collen MF, Catler JL, Siegelaub AB, et al.: Reliability of a self-administered medical questionnaire. Arch Int Med 123: 664681, 1969 Alexion NG, Wiener G: Reliability of a self-administered health questionnaire for secondary school students. Am J Pub1 Hlth 58: 1439-1446, 1968 Epstein LM: Validity of a questionnaire for diagnosis of peptic ulcer in an ethnically heterogeneous population. J Chron Dis 22: 49-57, 1970 Bernstein RA, Werner LH, Rimm AA: Relationship of gallbladder disease to parity, obesity and age: A study of 62,739 weight-conscious women. Hlth Sew Rep 88: 925-936, 1973 Rimm I, Rimm A: TOPS metabolic and obesity research: International research questionnaire. Prev Med 3 : 543-572, 1974 Kay H, et al.: SCALPEL:prototype for a computer assisted finite language for data screening. table method in obtaining clinically useful information. J Chron Dis: to be published Rimm AA, Werner LH, Bernstein RA: Disease and obesity in 73,532 women. Obesity/Bariatric Med 1: 77-84, 1972 Hershberg PI: Medical diagnosis: The role of a brief open-ended medical history questionnaire. J Med Educ 44: 293-297, 1969 Brodman K, Erdmann AJ, Jr, Lorge I, et al.: Cornell Medical Index-health questionnaire: As diagnostic instrument. JAMA 145: 152-157, 1951 Hutchison GB, Shapiro S, Densen PM: Evaluation of a mailed questionnaire. Am J Pub Hlth 52: 1894-1917, 1962 Madow W: Net differences in interview data on chronic conditions and information derived from medical records. Vital Health Stat, Series 2, 57: 1-9, 1973 Friedman GO, Kannel WB, Danter TR: The epidemiology of gallbladder disease: Observation in the Framingham Study. J Chron Dis 19: 273-292, 1966 Bernstein RA, Giefer EE, Werner LH, et al: Gallbladder disease-II. Utilization the of life table method.in obtaining clinically useful information. J Chron Dis: to be published Fleiss JL: Statistical Methods for Rates and Proportions, pp. 141-143. New York: John Wiley 1973 Nelson BD, Powaznik J, Benfield JR: Gallbladder disease in southwestern American Indians. Arch Surg 103: 41-43, 1971 APPENDIX
Procedurefor the calculation of estimates for Table 3 To estimate the number of women who incorrectly reported gallbladder sum of estimates from the study of the two groups of 50 patients each: Group I: 0.02 x 6328= 127 Group II: 0.32 x 3527= 1129
disease, we can use the
1256. The estimate of women who correctly reported gallbladder
disease is, thus, calculated to be:
(6328-127)+(3527-1129)=8599. If we assume a rate of 1.6% underreporting, which is probably conservative since it has been shown above that the physician questionnaire was relatively invalid, an estimate for the total number of women who underreported can be calculated as follows: 0.016x (62,739-9855)=846. Similarly the last value in the table, the estimates for number of women answering gallbladder disease ‘No’ correctly can be calculated to be: 52,884-846=52,038.