- Email: [email protected]
Validity of the Composite International Diagnostic Interview (CIDI) psychosis module in a psychiatric setting
\ PERGAMON
Journal of Psychiatric Research 21 "0887# 250Ð257
Validity of the Composite International Diagnostic Interview "CIDI# psychosis module in a psychiatric setting Lucy Coopera\ Lorna Petersa\ Gavin Andrewsa\b a
World Health Or`anization Collaboratin` Centre for Mental Health and Substance Abuse\ St Vincent|s Hospital\ Darlin`hurst\ New South Wales 1909\ Australia b School of Psychiatry\ University of New South Wales\ Australia Received 0 September 0886^ accepted 7 April 0887
Abstract This study aimed to test the procedural validity of the psychosis module of the Composite International Diagnostic Interview "CIDI# by comparing it with diagnostic checklists completed by experienced clinicians[ Seventy!_ve subjects were interviewed using the interviewer!administered version of the CIDI[ Their clinician"s# then completed diagnostic checklists for DSMIV and ICD09 diagnoses of schizophrenia[ Agreement was measured at the diagnostic\ criterion and subcriterion levels[ The validity standard "diagnostic checklist# was shown to be reliable with interrater agreement between the clinicians for the diagnosis of schizophrenia being excellent "k 9[71 for DSMIV and k 9[60 for ICD09#[ The agreement between the CIDI and the clinician checklists varied with sensitivities for DSMIV subcriteria ranging from 9[07 "negative symptoms# to 9[82 "bizarre delusions# and speci_cities ranging from 9[27 "catatonia# to 9[84 "disorganised speech#[ A similar pattern was found for ICD09 subcriteria] sensitivity varied from 9[08 "neologisms# to 9[89 "persistent delusions# and speci_city varied from 9[28 "catatonia# to 9[84 "negative symptoms#[ The poorest levels of agreement were found for symptoms requiring interviewer judgement[ The CIDI showed good agreement with clinician checklist diagnoses when the criteria were based on questions asked of the subjects[ When the interviewer was required to make judgement of behaviours\ the agreement between the CIDI and the clinician checklists was lower\ resulting in overall poor agreement between the CIDI and the clinician checklists[ Suggestions for improving the validity of the psychosis module of the CIDI are made[ Þ 0887 Elsevier Science Ltd[ All rights reserved[ Key words] Composite International Diagnostic Interview^ Structured diagnostic interviews^ Psychosis^ Validity
0[ Introduction The CIDI is a fully standardised structured diagnostic interview for the assessment of mental disorders[ It was developed by the World Health Organisation "WHO# in collaboration with the former US Alcohol\ Drug Abuse and Mental Health Administration "ADAMHA#[ Unlike its predecessor\ the NIMH!Diagnostic Interview Sched! ule "DIS\ Robins et al[\ 0870#\ it provides a range of diagnoses according to the de_nitions and criteria of both ICD09 Diagnostic Criteria for Research "WHO\ 0882# and the Diagnostic and Statistical Manual of Mental Disorders "DSMIV^ American Psychiatric Association\ 0883#[ To ensure concordance with current diagnostic systems\ the CIDI is continually updated and revised by a team of experts from around the world[ Version 1[0\ modi_ed to meet DSMIV criteria\ was released in 0886[
Corresponding author[ Tel[ ] 99501 8221 0902 ^ fax ] 99501 8221 3205 ^ e!mail ] gavinaÝcrufad[unsw[edu[au
As well as a twelve!month and lifetime version\ a fully computerised version of the interview is also available "CIDI!Auto^ WHO\ 0886#[ CIDI!Auto faithfully repro! duces the questions from the paper and pencil version of the CIDI\ with the skip patterns and probe!~ow chart questions implemented by the program[ It is available in two versions] interviewer!administered or respondent! administered[ Studies comparing CIDI!Auto with paper and pencil CIDI show the concordance between these instruments to be high "Peters et al[ in press#[ A main advantage of the CIDI is that it is designed to be administered by lay interviewers with no clinical training[ As all diagnoses are computed by the program\ using scoring algorithms based on ICD09 and DSMIV criteria\ no clinical judgment on the part of the inter! viewer is required[ Further\ the highly structured format of the CIDI ensures that sources of unreliability such as criterion variance\ information variance\ interpretation variance and observer variance are minimised "Spitzer\ 0872^ Wittchen\ 0883#[ Indeed\ a number of studies have shown the CIDI to have good to excellent reliability
9911Ð2845:87 ,08[99 Þ 0887 Published by Elsevier Science Ltd[ All rights reserved PII] S 9 9 1 1 Ð 2 8 4 5 " 8 7 # 9 9 9 1 0 Ð 0
251
L[ Cooper et al[:Journal of Psychiatric Research 21 "0887# 250Ð257
across raters\ cultures and time "e[g[ Andrews et al[\ 0884^ Wittchen et al[\ 0880^ for a review\ see Wittchen\ 0883#[ While the validity of the CIDI has been less extensively studied\ it is acceptable "Peters and Andrews\ 0884^ Wittchen\ 0883#[ The validity of the psychosis module\ however\ is less clearly established[ Studies investigating earlier versions of the module from the DIS "e[g[ Burnam et al[\ 0872^ Hendricks et al[\ 0872^ Pulver and Carpenter\ 0872^ Spengler and Wittchen\ 0877^ Wittchen et al[\ 0874# were disappointing with the DIS tending to under! estimate the prevalence of schizophrenia when measured against a range of clinical standards including chart diag! nosis "Hendricks et al[\ 0872^ DIS!DSM!III!R^ Erdman et al[\ 0876#^ LEAD "Longitudinal\ Expert\ All Data# standard "Wittchen et al[\ 0874# and the Present State Examination "Pulver and Carpenter\ 0872#[ The most recent study was of CIDI!Auto 0[0 and reported a poor result with a sensitivity for the diagnosis of schizophrenia of 9[35 "Rosenman et al[\ 0886#[ Methodological prob! lems\ such as the use of an ureliable validity standard "unstructured diagnosis of a single clinician# and the use of the respondent administered version of CIDI!Auto in a severely disturbed population\ warrants caution in drawing _rm conclusions from this result[ With the poor performance of the psychosis module and the lack of research investigating current versions of the module\ the aim of the present research was to exam! ine the procedural validity "cf[ Spitzer and Williams\ 0879# of the psychosis module of CIDI 1[0[ This is par! ticularly crucial given the current debate surrounding the value of using lay administered\ structured diagnostic instruments for the assessment of more severe mental disorders such as psychosis "Pulver and Carpenter\ 0872^ Spengler and Wittchen\ 0877#[ Validity was tested by examining the concordance of the CIDI with the struc! tured diagnoses of two independent clinicians[
1[ Method 1[0[ Subjects A total of 64 subjects "14 females\ 49 males^ age] mean 24[62\ S[D 02[82# were administered the psy! chosis module of CIDI!Auto 1[0[ A clinical sample was procured in order to overcome the low base rate of schizo! phrenia in the general population[ Subjects were selected who met the following criteria] "a# they could attend a 29!min interview and "b# they could give coherent responses to questions[ Interviews from two subjects had to be discarded from the _nal analysis because of com! prehension di.culties[ There were 59 patients from an acute psychiatric ward\ but due to the slow turnover of patients on the ward\ additional subjects were recruited from other clinical services[
1[1[ Desi`n ICD09 and DSMIV diagnoses of schizophrenia made by the computerised version of CIDI 1[0 were compared to ICD09 and DSMIV diagnoses made by clinicians using checklists for schizophrenia[ It was expected that adding structure to the diagnostic process would substantially improve the reliability of clinicians| diagnoses by limiting sources of observer\ information and criterion variance "cf[ Dohrenwend\ 0889^ McGorry et al[\ 0881^ Robins et al[\ 0877^ Spitzer\ 0872^ Wittchen\ 0883#[ Further\ explicitly listing the diagnostic criteria allowed a direct comparison at the criterion level between the CIDI and clinician diagnoses[
1[1[0[ CIDIÐauto psychosis module The psychosis module from the 01!month version of CIDI!Auto 1[0 was used[ The 01!month and lifetime versions of the CIDI are identical except for the time frames in which questions are phrased[ While the lifetime CIDI asks the respondent if they have ever experienced a symptom in their lifetime\ the 01!month CIDI asks if they have experienced the symptom in the past 01 months[ In other words\ in the 01!month CIDI a diagnosis is established only on the basis of the individuals| status during 01 months prior to interview[ CIDI!Auto is a complete computerised replication of the paper and pen! cil CIDI "WHO\ 0886#[ By following the skip rules and taking the interviewer through the correct route in the probe ~ow chart\ the opportunity for interviewer error is further reduced[ Internal scoring algorithms within the program provide DSMIV and ICD09 diagnoses for schizophrenia and related disorders[ Due to the severity of mental disorders in the sample\ the interviewer admin! istered\ as opposed to the respondent administered ver! sion of CIDI!Auto was used[ The psychosis module is one of the 04 discrete modules in the CIDI[ It can either be administered as part of the larger interview\ or by itself[ The psychosis module consists of 35 questions relating to ICD09 and DSMIV diagnostic criteria[ 24 of these questions are asked directly of the respondent and ask about delusional beliefs "e[g[ {{In the past 01 months\ have you ever believed people were spying on you<||# and hallucinatory experiences "e[g[ {{In the past 01 months\ have you more than once heard things other people couldn|t hear\ such as a voice<||#[ Respondents are then asked for an example which the interviewer judges for its plausibility[ The _nal 00 questions of the module are not asked of the respondent but are completed by the interviewer at the end of the interview[ These questions relate to the presence of symptoms such as hallucinatory behaviour\ catatonia\ brief empty speech and ~at a}ect[ Judgements are made on the basis of the respondent|s behaviour during the interview[
L[ Cooper et al[:Journal of Psychiatric Research 21 "0887# 250Ð257
1[1[1[ ICD09 and DSMIV checklists Clinicians| diagnoses were established using two sep! arate diagnostic checklists] one listing DSMIV diagnostic criteria for schizophrenia and the other ICD09 diagnostic criteria[ Next to each criterion or sub!criterion\ clinicians circled one of three possible response options] "0#\ if the symptom was not present^ "4#\ if the symptom was present and "8#\ if the clinician was uncertain[ While these check! lists were generated for the purpose of the current study\ criteria were listed using the exact wording from each classi_cation system[
1[2[ Procedure The CIDI interviews were conducted in private[ Written informed consent was obtained from all subjects before the interviews began[ Subjects were paid ,19 for their participation[ The two CIDI interviewers were four! year trained psychology graduates who had recently com! pleted a one!week intensive training course in admin! istering the CIDI at the WHO designated training centre in Sydney[ Given the high proportion of questions requir! ing interviewer judgements\ the _rst six interviews were rated by both interviewers[ This allowed rating dis! crepancies to be resolved in the initial stages[ To enable interrater reliability to be calculated across all the inter! views\ a further three interviews were double rated later in the study[ Average CIDI psychosis module interview time was 13 min "S[D[ 01 min#[ The treating clinicians were given both the ICD09 and DSMIV checklists to complete in the days after each CIDI interview had been completed[ Clinicians were not given access to the information obtained from the CIDI interview[ Where two treating clinicians were available "n 32#\ both clinicians completed the checklists[ Where there were discrepancies between the checklists\ the clin! icians met\ resolved the discrepancies and completed a third\ consensus checklist which was used as the validity standard[ This procedure generated a more reliable val! idity standard while also enabling the reliability of the clinicians| diagnoses to be assessed[ Eight clinicians par! ticipated "senior psychiatry residents\ consultant psy! chiatrists\ and clinical psychologists#[ All clinicians had extensive experience in the treatment and assessment of psychiatric disorders[ After all data were collected\ the WHO scoring pro! gram for the CIDI was used to generate ICD09 and DSMIV diagnoses[ As the entire CIDI was not admin! istered in the study\ some of the exclusion criteria for schizophrenia "e[g[ the presence of other mental dis! orders# could not be assessed and\ therefore\ were not included in the _nal diagnoses[ Clinician completed checklists were scored according to ICD09 and DSMIV criteria "excluding the criteria not addressed in the CIDI#[ As {uncertain| was not a response option in the CIDI\ all
252
8s were recoded as 0 "not present# in the _nal analysis^ i[e[ the more conservative estimate was used[ 1[3[ Analysis To measure interrater reliability between the clinicians\ Cohen|s kappas "0859# were calculated as a measure of chance!corrected agreement[ Interpretation of kappa was based on Fleiss "0870#[ To measure the concordance between the CIDI and the checklists\ percent agreement and sensitivity and speci_city statistics were calculated[ Statistics were cal! culated at the level of subcriterion\ criterion and diag! nostic level for both ICD09 and DSMIV diagnostic criteria[ Sensitivity measures the proportion of subjects positively diagnosed by the clinicians also positively diag! nosed by the CIDI[ Speci_city measures the proportion of subjects not diagnosed by the clinicians with schizo! phrenia who are also not diagnosed by the CIDI[ While the kappa statistic is often calculated in studies to assess validity\ several factors make sensitivity and speci_city more appropriate statistical measures for this design "see Maclure and Willett\ 0876 for a review#[ Firstly\ kappa assumes that the same instrument is being employed by the raters on both occasions[ In other words\ kappa treats the diagnostic standard "clinicians| diagnosis# as identical to the instrument being validated "CIDI#[ This makes kappa more a measure of reliability rather than validity "Anthony et al[\ 0874^ Helzer et al[\ 0874^ Robins\ 0874#[ Secondly\ kappa is sensitive to sample characteristics such as prevalence rates[ This limits the generalizability of the results of the study and renders comparisons between studies uninformative and misleading "Fleiss\ 0870#[ Thus\ kappa was not used as a statistical measure of validity in this study[
2[ Results 2[0[ CIDI interrater reliability The nine double rated CIDI interviews were inves! tigated for any discrepancies[ For individual criteria across the interviews\ only two had more than one dis! crepancy[ These were catatonic behaviour and delusions\ both coded using interviewer judgements[ 2[1[ Clinician checklist interrater reliability To assess the integrity of the clinicians| diagnoses\ kappas were calculated at the sub!criterion\ criterion and diagnostic level for both ICD09 and DSMIV diagnostic criteria[ The agreement between the clinicians ranged from good to excellent "see Table 0#\ with most kappas above 9[64[ Kappa for agreement between the clinicians for the diagnosis of schizophrenia was 9[71 for DSMIV
253
L[ Cooper et al[:Journal of Psychiatric Research 21 "0887# 250Ð257 Table 0 Interrater reliability between clinicians on the DSMIV and ICD09 checklists for the diagnosis of schizophrenia DSMIV Criteria
Kappa
ICD!09 General criteria
Kappa
184[XX Diagnosis of schizophrenia Criterion A A0 Delusions Bizarre delusions A1 Hallucinations A2 Disorganised speech A3 Catatonia A4 Negative symptoms Criterion B ] social:occupational Criterion C ] duration Criterion E ] substance:medical exclusions
9[71 9[73 9[74 9[70 9[76 9[75 9[64 9[85 9[62 9[89 9[64
F19[X Diagnosis of schizophrenia Criterion G0 0A Thought echo 0B Control delusions 0C Hallucinatory voices 0D Persistent delusions 1A Persistent hallucinations 1B Neologisms 1C Catatonia 1D Negative symptoms Criterion G1[1 ] substance:medical exclusions
9[60 9[70 9[70 9[79 9[73 9[67 9[80 9[74 9[70 9[58 9[69
and 9[60 for ICD09[ These results indicate the validity standard to be reliable[ 2[2[ Clinician checklists vs CIDI Amongst the 62 subjects\ the CIDI made 6 DSMIV diagnoses of schizophrenia and 03 ICD09 diagnoses of schizophrenia[ This was considerably fewer than the clin! icians who made 39 DSMIV diagnoses and 23 ICD09 diagnoses of schizophrenia[ Table 1 presents the sen! sitivity and speci_city scores and percent agreement for the CIDI at the subcriterion\ criterion and diagnostic level for ICD09 and DSMIV[ Firstly\ for DSMIV criteria\ the performance of the CIDI was mixed[ Percent agree! ment was 49[6 for the diagnosis of schizophrenia\ and ranged between 32[6 and 66[4 at the criterion level[ Sensitivity ranged from very good "9[82 bizarre delusions#\ to very poor "9[07 negative symptoms#[ Over! all\ speci_city scores were better\ but still varied "range
9[27Ð9[84#[ In general\ the sensitivity of the CIDI was particularly poor for the sub criteria scored at the end of the interview using interviewer judgements] disorganised speech\ negative symptoms and catatonia\ resulting in poor sensitivity for General Criterion A[ The sensitivity for the diagnosis of schizophrenia was very low "9[04#\ with high speci_city "9[86#[ For ICD09 criteria\ the pattern of results is repeated[ Agreement was 46[7) for the diagnosis of schizophrenia and ranged between 31[2 and 66[4) at the criterion level[ Sensitivity varied considerably from low "9[08 neol! ogisms# to high "9[89 persistent delusions#[ Speci_city also varied but was higher "range 9[28Ð9[84#[ Again\ the sensitivity of the CIDI was consistently poor for the sub criteria scored at the end of the interview using the interviewer judgements\ neologisms and negative symp! toms\ resulting in poor sensitivity for General Criterion 0[ The sensitivity for the diagnosis of schizophrenia was again low "9[15# and speci_city\ high "9[75#[
Table 1 CIDI vs clinician checklists ] percentage agreement\ sensitivity and speci_city for DSMIV and ICD09 at the diagnostic\ criterion and sub criterion level
DSMIV criteria 184[XX Schizophrenia Criterion A A0 Delusions Bizarre delusions A1 Hallucinations A2 Disorganised speech A3 Catatonia A4 Negative symptoms Criterion B ] social:occupational Criterion C ] duration Criterion E ] substance:medical exclusions
) agreement Sensitivity Speci_city ICD!09 criteria 49[6 41[0 66[4 62[1 56[5 53[7 34[0 52[3 55[1 32[6 55[1
9[04 9[26 9[82 9[66 9[75 9[10 9[52 9[07 9[57 9[19 9[57
9[86 9[75 9[44 9[56 9[49 9[84 9[27 9[82 9[47 9[84 9[52
F19[X Schizophrenia Criterion G0 0A Thought echo 0B Control delusions 0C Hallucinatory voices 0D Persistent delusions 1A Persistent hallucinations 1B Neologisms 1C Catatonia 1D Negative symptoms Criterion G1[1 ] substance:medical exclusions
) agreement Sensitivity Speci_city 46[7 41[0 42[4 69[3 69[3 66[4 48[1 55[1 31[2 52[3 55[1
9[15 9[24 9[71 9[56 9[59 9[89 9[48 9[08 9[56 9[19 9[60
9[75 9[73 9[37 9[61 9[64 9[52 9[59 9[82 9[28 9[84 9[59
L[ Cooper et al[:Journal of Psychiatric Research 21 "0887# 250Ð257
3[ Discussion This study presents data on the validity of the psychosis module of the interviewer administered CIDI!Auto "1[0#[ Validity was assessed by examining the concordance between the CIDI!Auto as administered by trained\ lay interviewers and diagnostic checklists completed by clin! icians[ Unlike previous validity studies on the CIDI "e[g[ Janca et al[\ 0881^ Kovess et al[\ 0881#\ the validity stan! dard of this study was not contaminated by allowing the clinicians access to the information obtained in the CIDI interview[ Further\ as can be seen from the good inter! rater reliability between the clinicians\ the structured for! mat of the checklists provided a reliable validity standard[ Overall\ results from the study suggest that compared to clinicians| diagnoses\ the psychosis module of the CIDI has good speci_city\ but poor sensitivity[ For DSMIV criteria the sensitivity and speci_city for the diagnosis of schizophrenia was 9[04 and 9[86 respectively and for ICD09 criteria\ the sensitivity and speci_city was 9[15 and 9[75 respectively[ This suggests that while the CIDI was accurate in the positive diagnoses it made\ it had a high {miss| rate\ failing to detect many of the positive diagnoses made by the clinicians[ Compared to the clin! icians\ the CIDI detected only 07) of DSMIV diagnoses of schizophrenia\ and 30) of ICD09 diagnoses of schizo! phrenia resulting in a considerable underestimation of the {true| prevalence rate in this sample[ These _ndings are in accordance with earlier studies investigating the validity of the psychosis module of the DIS[ Results from these studies show the psychosis mod! ule to considerably underestimate the prevalence of schizophrenia in both general and clinical populations "Wittchen et al[\ 0874#\ for patients in both active and non!active phases of the disorder "Semler et al[\ 0876^ Spengler and Wittchen\ 0877^ Wittchen et al[\ 0874#\ and for lifetime and current diagnosis "Spengler and Wittchen\ 0877#[ In the study by Wittchen et al[ "0874#\ the psychosis module had the poorest concordance with structured clinical diagnoses "LEAD standard# out of all the DIS modules[ Similarly\ in a study which examined the validity of the DIS when administered by trained lay interviewers\ the DIS failed to detect one third of subjects with a well documented history of the disorder "Pulver and Carpenter\ 0877#[ While the considerable time delay of six years between the clinicians| diagnoses and DIS interview necessitates caution in interpreting this par! ticular result\ it is consistent with the general _nding that the psychosis module considerably underestimates prevalence of the disorder[ Finally\ in the more recent study by Rosenman et al[ "0886# using CIDI!Auto 0[0\ sensitivity for the ICD09 diagnosis of schizophrenia and related disorders was 9[35 and speci_city 9[75[ The lower sensitivities obtained in the present study "9[15# could re~ect the fact that the self!administered version of CIDI! Auto does not include the interviewer!rated questions at
254
the end of the interview which generally performed poorly "e[g[ neologisms\ negative symptoms# and this utilised a less stringent validity standard[ While variations in sample characteristics and method! ologies between these previous studies and the current study necessitate caution when making comparisons\ the similar pattern in results suggests particular problems with the validity of the psychosis module[ An advantage of the current study was that the validity standard enabled a more sensitive analysis of the performance of the CIDI at the criterion and sub criterion levels[ The performance of the CIDI varied considerably between individual criteria[ For ICD09 criteria\ sensitivities ranged between 9[08 "neologisms# and 9[89 "persistent delusions#\ and for DSMIV criteria sensitivities ranged between 9[07 "negative symptoms# and 9[82 "bizarre delusions#[ For both diagnostic systems\ the sensitivities for 5 out of the 09 criteria were 9[59 or above[ This suggests that the low sensitivity obtained for the diagnosis of schizophrenia may be attributable to the poor per! formance of the CIDI on a small number of criteria[ For DSMIV\ the poorly performing criteria were dis! organised speech\ duration greater than six months\ and negative symptoms "sensitivity 9[10^ 9[19 and 9[07 respec! tively#\ and for ICD09 they were neologisms and negative symptoms "sensitivity 9[08 and 9[19 respectively#[ Apart from duration for DSMIV\ all these poorly performing criteria are scored from the _nal 00 questions rated by the interviewer at the end of the interview[ These require the interviewer to judge\ for example\ if the subject is able to {persist in goal directed activities|\ or whether they exhibit features of {~at a}ect|\ or {thought disorder[| Two explanations could account for the low concordance between the CIDI interviewers and clinicians for these items[ Firstly\ it is possible that only administering the psychosis module of the CIDI did not allow su.cient time for interviewers to observe the behaviours to be rated and hence\ did not allow the interviewers to make accurate judgements[ Secondly\ the sensitivity for these items may have been poor because they were assessed using the judgements of lay interviewers[ While other criteria are assessed in the module on the basis of subject self!report\ these criteria are assessed entirely on the basis of interviewer judgement[ Given the complex clinical nat! ure of these criteria\ it is probable that interviewers who do not have a clinical background do not possess the necessary clinical skills to make these judgements[ In support of this second explanation\ the psychosis module is the only module in the CIDI for which adequate validity has not yet been established "see Wittchen\ 0883# and it is the only module which depends on interviewer judgement to establish diagnoses[ For all other modules\ diagnoses are made solely on the basis of subject self!report[ A problem unique to the psychotic disorders\ however\ is that diagnostic criteria relating to facial expression and speech patterns can only be assessed
255
L[ Cooper et al[:Journal of Psychiatric Research 21 "0887# 250Ð257
by an outside observer[ In light of this restriction\ one means of improving the validity of this module would be to extend the current interviewer training program to focus speci_cally on the identi_cation and assessment of these criteria[ The better sensitivities obtained for the criteria relating to the more obvious symptoms of delusions and hallucinations would suggest that lay inter! viewers are able to distinguish between {normal| and clini! cally signi_cant experiences when provided with self! reported symptoms[ Alternatively\ how these criteria are currently described and operationalised in the CIDI could be improved[ Sensitivities for these items may be low simply because they have poor construct validity[ In other words\ interviewers are not coding these items incorrectly because they lack the necessary skills\ but because they are poorly conceptualised and described by the CIDI text[ A way to investigate this hypothesis would be to have trained\ experienced clinicians administer these items[ If poor sensitivities were still obtained for these items\ this would suggest problems in their oper! ationalisation as opposed to problems with the judge! ments of lay interviewers[ Other factors\ however\ are also likely to be con! tributing to the overall poor validity of this module[ While the tendency for structured diagnostic interviews in general to underdiagnose compared to clinicians has been well documented in the literature "e[g[ Fennig et al[\ 0883^ McGorry et al[\ 0889^ Wittchen et al[\ 0874#\ fea! tures of psychosis are considered to make its assessment via structured interviews particularly tenuous[ Notably\ the episodic nature of the illness is believed to make obtaining an accurate assessment from a single interview very di.cult\ with results ~uctuating depending upon the illness {phase| in which the interview takes place "Wittchen et al[\ 0878#[ In the non!active phase of the disorder\ fear of social stigmatisation associated with psy! chotic disorders is believed to lead subjects to either deny\ or minimise experiencing symptoms "Anthony et al[\ 0874^ Erdman et al[\ 0876^ Pulver and Carpenter\ 0877#[ This is evident in the particularly poor recall of symptoms shown by subjects with no ~orid symptoms at the time of interview "Wittchen et al[\ 0874#[ Then\ in the {active| phase of the disorder\ features of psychosis such as de_! cits in a}ect\ insight and reasoning interfere with the subject|s ability to give coherent and accurate responses "Farmer et al[\ 0876^ McGorry et al[\ 0878^ Pulver and Carpenter\ 0887#[ In support of this\ it has been shown that the accuracy of CIDI diagnoses notably decreases for cases in an active phase of schizophrenia "Semler et al[\ 0876#[ This is likely to have been particularly prob! lematic in the current study\ given that the sample came predominantly from an acutely ill population[ These di.culties associated with psychosis have often been raised as arguments against the use of structured interviews such as the CIDI for assessment of more
chronic disorders such as schizophrenia[ The problem with these self!report instruments is that their reliability and validity is ultimately dependent on the lucidity and compliance of the subject[ In the case of psychoses\ however\ this cannot be depended upon and testi_es to the need to include sources of information other than subject self!report to establish accurate diagnoses[ For instance\ information collected by the CIDI could be supplemented with information obtained from medical records or interviews with close relatives of the subject[ More generally\ however\ it should be noted that the endeavour to achieve perfect agreement with diagnoses made by clinicians is unrealistic[ Firstly clinicians are able to draw upon multiple sources of information and their own clinical experience in reaching diagnoses[ As well\ structured diagnostic interviews are more stringent in their interpretation and application of the diagnostic cri! teria meaning that compared to clinicians\ they will always tend to underdiagnose {true| prevalence rates[ Secondly\ the less than perfect agreement found between clinicians in the diagnosis of schizophrenia creates a {ceil! ing|] the validity of the instrument can only be as good as that of the diagnostic standard it is compared with[ Kappas of between 9[58 and 9[60 for example\ are gen! erally reported for the diagnosis of schizophrenia by two clinicians "Robins\ 0874^ Winokur et al[\ 0877#[ In this study\ while the structure of the checklists ensured the clinicians applied the criteria\ kappas for the diagnosis of schizophrenia were still only 9[71 for ICD09 criteria\ and 9[60 for DSMIV[ Hence\ until a gold standard in psychiatry is obtained\ the validity of instruments such as the CIDI will always be less than perfect[ Finally\ it should also be noted that while the CIDI was designed primarily as an epidemiological instrument for use in general populations\ as with most previous validity studies on the psychosis module\ the current study was conducted within a psychiatric setting[ Given the nature of psychosis\ it is likely that the way in which this module in particular performs in a clinical population will be di}erent to the way in which it performs in the general population[ Indeed\ a recent study by Kendler et al[ "0885# using a modi_ed University of Michigan ver! sion of the CIDI "UM!CIDI# to assess lifetime prevalence rates of disorders in the general population showed just this[ While the lifetime prevalence rate for schizo! phrenia:schizophreniform disorder was estimated to be 0[2) by the CIDI\ the equivalent prevalence rate based on clinicians| diagnoses was 9[1)[ Only 09) of the posi! tive diagnoses made by the CIDI were con_rmed by the clinicians[ In other words\ the psychosis module when used in the community considerably overdiagnosed the prevalence of schizophrenia and related disorders\ thereby reversing the trend found in clinical populations[ One explanation given for the high false positives found in the Kendler et al[ "0885# study was that the threshold for judging if a symptom is {psychotic| is too low in the
L[ Cooper et al[:Journal of Psychiatric Research 21 "0887# 250Ð257
CIDI[ This would indicate a problem with the plausibility judgements made by interviewers to assess the clinical signi_cance of {unusual| experiences[ The good sensitivity and speci_city found for these items in the current study\ however\ suggests that these judgements are only di.cult to make in general populations in which the symptoms are likely to be less obvious[ In conclusion\ the following directions for future research are recommended in order to improve the poor validity of the psychosis module[ Firstly\ the module|s current reliance on interviewer judgement at the end of the interview\ particularly in relation to the assessment of negative symptoms\ needs to be addressed[ Secondly\ using other sources of information to supplement the CIDI in establishing diagnoses\ particularly in a psy! chiatric setting\ needs to be considered[ This would also help to identify additional reasons for the module|s cur! rent high false negative rate[ Finally\ the way in which the module performs in less severely disturbed populations needs to be determined\ particularly in relation to its ability to discriminate threshold or subclinical cases from {true| cases[ This is particularly crucial given the CIDI|s primary role as an epidemiological instrument[
Acknowledgements This study was supported by a grant from the Com! monwealth of Australia\ Research and Development Grants Advisory Committee[
References American Psychiatric Association[ Diagnostic and statistical manual of mental disorders[ 3th ed[ Washington\ DC] American Psychiatric Association\ 0883[ Andrews G\ Peters L\ Guzman A\ Bird K[ A comparison of two struc! tured diagnostic interviews] CIDI and SCAN[ Australian and New Zealand Journal of Psychiatry 0884^18]013Ð21[ Anthony JC\ Folstein M\ Romanowski AJ\ von Kor} MR\ Nestadt GR\ Chadal R\ Merchant A\ Brown H\ Shapiro S\ Kramer M\ Gruenberg EM[ Comparison of the lay Diagnostic Interview Sched! ule and a standardized psychiatric diagnosis[ Experiences Eastern Baltimore[ Archives of General Psychiatry 0874^31]556Ð64[ Burnam MA\ Karno M\ Hough RL\ Escobar JI\ Forsyth AB[ The Spanish Diagnostic Interview Schedule[ Reliability and comparison with clinical diagnoses[ Archives of General Psychiatry 0872^39]0078Ð85[ Cohen\ J[ A coe.cient of agreement for nominal scales[ Educational Psychological Measurement 0859^19]26Ð35[ Dohrenwend BP[ {The problem of validity in _eld studies of psycho! logical disorders| revisited[ Psychological Medicine 0889^19]084Ð 197[ Erdman HP\ Klein MH\ Greist JH\ Bass SM\ Bires JK\ Machtinger PE[ A comparison of the Diagnostic Interview Schedule and clinical diagnosis[ American Journal of Psychiatry 0874^033]0366Ð79[ Farmer AE\ Katz R\ McGu.n P\ Bebbington P[ A comparison between the present state examination and the Composite International
256
Diagnostic Interview[ Archives of General Psychiatry 0876^33]0953Ð 8[ Fennig S\ Craig T\ Lavelle J\ Kovasznay B\ Bromet EJ[ Best!estimate versus structured interview!based diagnosis in _rst!admission psy! chosis[ Comprehensive Psychiatry 0883^24]230Ð7[ Fleiss JL[ Statistical methods for rates and proportions[ 1nd ed[ New York] John Wiley and Sons\ 0870[ Helzer JE\ Robins LN\ McEvoy LT[ A comparison of clinical and Diagnostic Interview Schedule diagnoses] physician reexamination of lay!interviewed cases in the general population[ Archives of Gen! eral Psychiatry 0874^31]546Ð55[ Hendricks LE\ Bayton JA\ Collins JL\ Mathura CB\ McMillan SR\ Montgomery TA[ The NIMH Diagnostic Interview Schedule] a test of its validity in a population of black adults[ Journal of National Medical Association 0872^64]556Ð60[ Janca A\ Robins LN\ Cottler LB\ Early TS[ Clinical observation of assessment using the Composite International Diagnostic Interview "CIDI#] an analysis of the CIDI _eld trials*wave II at the St Louis site[ British Journal of Psychiatry 0881^059]704Ð7[ Kendler KS\ Gallagher TJ\ Abelson JM\ Kessler RC[ Lifetime preva! lence\ demographic risk factors\ and diagnostic validity of non! a}ective psychosis as assessed in a US community sample[ Archives of General Psychiatry 0885^42^0911Ð20[ Kovess V\ Sylla O\ Fournier L\ Favigny V[ Why discrepancies exist between structured diagnostic interviews and clinicians| diagnoses[ Social Psychiatry and Psychiatric Epidemiology 0881^16]074Ð80[ Maclure M\ Willett WC[ Misinterpretation and misuse of the kappa statistic[ American Journal of Epidemiology 0876^015]050Ð8[ McGorry PD\ Copolov DL\ Singh BS[ Royal Park Multidiagnostic Instrument for Psychosis] part I[ Rational and review[ Schizo! phrenia Bulletin 0878^05]490Ð04[ McGorry PD\ Singh B\ Connell S\ McKenzie D\ Van Riel R\ Copolov DL[ Diagnostic concordance in functional psychosis revisited] a study of inter!relationships between alternative concepts of psy! chotic disorder[ Psychological medicine 0881^11]256Ð67[ McGorry PD\ Singh B\ Copolov DL\ Kaplan I\ Dossetor CR\ van Riel RJ[ Royal Park Multidiagnostic Instrument for Psychosis II[ Development\ reliability and validity[ Schizophrenia Bulletin 0889^05]406Ð25[ Peters L\ Andrews G[ Procedural validity of the computerized version of the Composite International Diagnostic Interview "CIDI!Auto# in anxiety disorders[ Psychological Medicine 0884^14]0158Ð79[ Peters L\ Clark D\ Carroll F[ Are computerised interviews equivalent to human interviewers< CIDI!Auto vs CIDI in anxiety and depressive disorders[ Psychological Medicine\ in press[ Pulver AE\ Carpenter WT[ Lifetime psychotic symptoms assessed with the DIS[ Schizophrenia Bulletin 0872^8]266Ð71[ Pulver AE\ Carpenter WT[ Accuracy of the diagnoses of a}ective dis! orders and schizophrenia in public hospitals[ American Journal of Psychiatry 0877^034]107Ð19[ Robins LN[ Re~ections on testing the validity of psychiatric interviews[ Archives of General Psychiatry 0874^31]807Ð13[ Robins LN\ Helzer JE\ Croughan J\ Ratcli} KS[ National Institute of Mental Health Diagnostic Interview Schedule[ Archives of General Psychiatry 0870^27]270Ð8[ Robins LN\ Wing J\ Wittchen H!U\ Helzer JE\ Babor TF\ Burke J\ Farmer A\ Jablenski A\ Pickens R\ Regier D\ Sartorius N\ Towle LH[ The Composite International Diagnostic Interview[ Archives of General Psychiatry 0877^34]0958Ð66[ Rosenman SJ\ Korten AE\ Levings CT[\ Computerised diagnosis in acute psychiatry] validity of CIDI!Auto against routine clinical diagnoses[ Journal of Psychiatry Research 0886^20]470Ð81[ Semler G\ Wittchen H!U\ Joschke K\ Zaudig M\ von Geiso T\ Kaiser S\ von Cranach M\ P_ster H[ Test!retest reliability of a standardized psychiatric interview "DIS:CIDI#[ European Archives of Psychiatry and Neurological Sciences 0876^125]103Ð11[ Spengler PA\ Wittchen H!U[ Procedural validity of standardized symp!
257
L[ Cooper et al[:Journal of Psychiatric Research 21 "0887# 250Ð257
tom questions for the assessment of psychotic symptoms] A com! parison of the DIS with two clinical methods[ Comprehensive Psy! chiatry 0877^18]298Ð11[ Spitzer RL[ Psychiatric diagnosis] are clinicians still necessary< Com! prehensive Psychiatry 0872^13]288Ð300[ Spitzer RL\ Williams JBW[ Procedural validity] validity of the diag! nostic process[ In Kaplan HI\ Freedman AM\ Sadock BJ\ editors[ Comprehensive Textbook of Psychiatry 2rd ed[ Baltimore] Williams and Wilkins Company\ 0879[ Winokur G\ Zimmerman M\ Cadoret R[ Cause the bible tells me so[ Archives of General Psychiatry 0877^34]572Ð3[ Wittchen\ H[ [ Reliability and validity studies of the WHO!Composite International Diagnostic Interview "CIDI#] a critical review[ Journal of Psychiatric Research 0883^17]46Ð3[ Wittchen H!U\ Semler G\ von Zerssen D[ A comparison of two diag! nostic methods[ Clinical ICD diagnosis vs DSM!III and RDC using the DIS "version II#[ Archives of General Psychiatry 0874^31]566Ð 73[
Wittchen H!U\ Burke JD\ Semler G\ P_ster H\ von Cranach M\ Zaudig M[ Recall and dating of psychiatric symptoms] test!retest reliability of time!related symptom questions in a standardized psychiatric interview[ Archives of General Psychiatry 0878^35]326Ð32[ Wittchen H!U\ Robins LN\ Cottler LB\ Sartorius N\ Burke JD\ Regier D[ Participants in the multicentre WHO:ADAMHA _eld trials[ Cross!cultural feasibility\ reliability and sources of variance of the Composite International Diagnostic Interview "CIDI#[ British Jour! nal of Psychiatry 0880^048]534Ð42[ World Health Organization[ The ICD09 classi_cation of mental and behavioral disorders] diagnostic criteria for research[ Geneva\ WHO\ 0882[ World Health Organization[ CIDI!Auto "1[0# administrator|s guide and reference[ Sydney] Fast Books\ 0886[ Zimmerman M[ Why are we rushing to publish DSM!IV[ Archives of General Psychiatry 0877^34]0024Ð7[
Journal of Psychiatric Research 21 "0887# 250Ð257
Validity of the Composite International Diagnostic Interview "CIDI# psychosis module in a psychiatric setting Lucy Coopera\ Lorna Petersa\ Gavin Andrewsa\b a
World Health Or`anization Collaboratin` Centre for Mental Health and Substance Abuse\ St Vincent|s Hospital\ Darlin`hurst\ New South Wales 1909\ Australia b School of Psychiatry\ University of New South Wales\ Australia Received 0 September 0886^ accepted 7 April 0887
Abstract This study aimed to test the procedural validity of the psychosis module of the Composite International Diagnostic Interview "CIDI# by comparing it with diagnostic checklists completed by experienced clinicians[ Seventy!_ve subjects were interviewed using the interviewer!administered version of the CIDI[ Their clinician"s# then completed diagnostic checklists for DSMIV and ICD09 diagnoses of schizophrenia[ Agreement was measured at the diagnostic\ criterion and subcriterion levels[ The validity standard "diagnostic checklist# was shown to be reliable with interrater agreement between the clinicians for the diagnosis of schizophrenia being excellent "k 9[71 for DSMIV and k 9[60 for ICD09#[ The agreement between the CIDI and the clinician checklists varied with sensitivities for DSMIV subcriteria ranging from 9[07 "negative symptoms# to 9[82 "bizarre delusions# and speci_cities ranging from 9[27 "catatonia# to 9[84 "disorganised speech#[ A similar pattern was found for ICD09 subcriteria] sensitivity varied from 9[08 "neologisms# to 9[89 "persistent delusions# and speci_city varied from 9[28 "catatonia# to 9[84 "negative symptoms#[ The poorest levels of agreement were found for symptoms requiring interviewer judgement[ The CIDI showed good agreement with clinician checklist diagnoses when the criteria were based on questions asked of the subjects[ When the interviewer was required to make judgement of behaviours\ the agreement between the CIDI and the clinician checklists was lower\ resulting in overall poor agreement between the CIDI and the clinician checklists[ Suggestions for improving the validity of the psychosis module of the CIDI are made[ Þ 0887 Elsevier Science Ltd[ All rights reserved[ Key words] Composite International Diagnostic Interview^ Structured diagnostic interviews^ Psychosis^ Validity
0[ Introduction The CIDI is a fully standardised structured diagnostic interview for the assessment of mental disorders[ It was developed by the World Health Organisation "WHO# in collaboration with the former US Alcohol\ Drug Abuse and Mental Health Administration "ADAMHA#[ Unlike its predecessor\ the NIMH!Diagnostic Interview Sched! ule "DIS\ Robins et al[\ 0870#\ it provides a range of diagnoses according to the de_nitions and criteria of both ICD09 Diagnostic Criteria for Research "WHO\ 0882# and the Diagnostic and Statistical Manual of Mental Disorders "DSMIV^ American Psychiatric Association\ 0883#[ To ensure concordance with current diagnostic systems\ the CIDI is continually updated and revised by a team of experts from around the world[ Version 1[0\ modi_ed to meet DSMIV criteria\ was released in 0886[
Corresponding author[ Tel[ ] 99501 8221 0902 ^ fax ] 99501 8221 3205 ^ e!mail ] gavinaÝcrufad[unsw[edu[au
As well as a twelve!month and lifetime version\ a fully computerised version of the interview is also available "CIDI!Auto^ WHO\ 0886#[ CIDI!Auto faithfully repro! duces the questions from the paper and pencil version of the CIDI\ with the skip patterns and probe!~ow chart questions implemented by the program[ It is available in two versions] interviewer!administered or respondent! administered[ Studies comparing CIDI!Auto with paper and pencil CIDI show the concordance between these instruments to be high "Peters et al[ in press#[ A main advantage of the CIDI is that it is designed to be administered by lay interviewers with no clinical training[ As all diagnoses are computed by the program\ using scoring algorithms based on ICD09 and DSMIV criteria\ no clinical judgment on the part of the inter! viewer is required[ Further\ the highly structured format of the CIDI ensures that sources of unreliability such as criterion variance\ information variance\ interpretation variance and observer variance are minimised "Spitzer\ 0872^ Wittchen\ 0883#[ Indeed\ a number of studies have shown the CIDI to have good to excellent reliability
9911Ð2845:87 ,08[99 Þ 0887 Published by Elsevier Science Ltd[ All rights reserved PII] S 9 9 1 1 Ð 2 8 4 5 " 8 7 # 9 9 9 1 0 Ð 0
251
L[ Cooper et al[:Journal of Psychiatric Research 21 "0887# 250Ð257
across raters\ cultures and time "e[g[ Andrews et al[\ 0884^ Wittchen et al[\ 0880^ for a review\ see Wittchen\ 0883#[ While the validity of the CIDI has been less extensively studied\ it is acceptable "Peters and Andrews\ 0884^ Wittchen\ 0883#[ The validity of the psychosis module\ however\ is less clearly established[ Studies investigating earlier versions of the module from the DIS "e[g[ Burnam et al[\ 0872^ Hendricks et al[\ 0872^ Pulver and Carpenter\ 0872^ Spengler and Wittchen\ 0877^ Wittchen et al[\ 0874# were disappointing with the DIS tending to under! estimate the prevalence of schizophrenia when measured against a range of clinical standards including chart diag! nosis "Hendricks et al[\ 0872^ DIS!DSM!III!R^ Erdman et al[\ 0876#^ LEAD "Longitudinal\ Expert\ All Data# standard "Wittchen et al[\ 0874# and the Present State Examination "Pulver and Carpenter\ 0872#[ The most recent study was of CIDI!Auto 0[0 and reported a poor result with a sensitivity for the diagnosis of schizophrenia of 9[35 "Rosenman et al[\ 0886#[ Methodological prob! lems\ such as the use of an ureliable validity standard "unstructured diagnosis of a single clinician# and the use of the respondent administered version of CIDI!Auto in a severely disturbed population\ warrants caution in drawing _rm conclusions from this result[ With the poor performance of the psychosis module and the lack of research investigating current versions of the module\ the aim of the present research was to exam! ine the procedural validity "cf[ Spitzer and Williams\ 0879# of the psychosis module of CIDI 1[0[ This is par! ticularly crucial given the current debate surrounding the value of using lay administered\ structured diagnostic instruments for the assessment of more severe mental disorders such as psychosis "Pulver and Carpenter\ 0872^ Spengler and Wittchen\ 0877#[ Validity was tested by examining the concordance of the CIDI with the struc! tured diagnoses of two independent clinicians[
1[ Method 1[0[ Subjects A total of 64 subjects "14 females\ 49 males^ age] mean 24[62\ S[D 02[82# were administered the psy! chosis module of CIDI!Auto 1[0[ A clinical sample was procured in order to overcome the low base rate of schizo! phrenia in the general population[ Subjects were selected who met the following criteria] "a# they could attend a 29!min interview and "b# they could give coherent responses to questions[ Interviews from two subjects had to be discarded from the _nal analysis because of com! prehension di.culties[ There were 59 patients from an acute psychiatric ward\ but due to the slow turnover of patients on the ward\ additional subjects were recruited from other clinical services[
1[1[ Desi`n ICD09 and DSMIV diagnoses of schizophrenia made by the computerised version of CIDI 1[0 were compared to ICD09 and DSMIV diagnoses made by clinicians using checklists for schizophrenia[ It was expected that adding structure to the diagnostic process would substantially improve the reliability of clinicians| diagnoses by limiting sources of observer\ information and criterion variance "cf[ Dohrenwend\ 0889^ McGorry et al[\ 0881^ Robins et al[\ 0877^ Spitzer\ 0872^ Wittchen\ 0883#[ Further\ explicitly listing the diagnostic criteria allowed a direct comparison at the criterion level between the CIDI and clinician diagnoses[
1[1[0[ CIDIÐauto psychosis module The psychosis module from the 01!month version of CIDI!Auto 1[0 was used[ The 01!month and lifetime versions of the CIDI are identical except for the time frames in which questions are phrased[ While the lifetime CIDI asks the respondent if they have ever experienced a symptom in their lifetime\ the 01!month CIDI asks if they have experienced the symptom in the past 01 months[ In other words\ in the 01!month CIDI a diagnosis is established only on the basis of the individuals| status during 01 months prior to interview[ CIDI!Auto is a complete computerised replication of the paper and pen! cil CIDI "WHO\ 0886#[ By following the skip rules and taking the interviewer through the correct route in the probe ~ow chart\ the opportunity for interviewer error is further reduced[ Internal scoring algorithms within the program provide DSMIV and ICD09 diagnoses for schizophrenia and related disorders[ Due to the severity of mental disorders in the sample\ the interviewer admin! istered\ as opposed to the respondent administered ver! sion of CIDI!Auto was used[ The psychosis module is one of the 04 discrete modules in the CIDI[ It can either be administered as part of the larger interview\ or by itself[ The psychosis module consists of 35 questions relating to ICD09 and DSMIV diagnostic criteria[ 24 of these questions are asked directly of the respondent and ask about delusional beliefs "e[g[ {{In the past 01 months\ have you ever believed people were spying on you<||# and hallucinatory experiences "e[g[ {{In the past 01 months\ have you more than once heard things other people couldn|t hear\ such as a voice<||#[ Respondents are then asked for an example which the interviewer judges for its plausibility[ The _nal 00 questions of the module are not asked of the respondent but are completed by the interviewer at the end of the interview[ These questions relate to the presence of symptoms such as hallucinatory behaviour\ catatonia\ brief empty speech and ~at a}ect[ Judgements are made on the basis of the respondent|s behaviour during the interview[
L[ Cooper et al[:Journal of Psychiatric Research 21 "0887# 250Ð257
1[1[1[ ICD09 and DSMIV checklists Clinicians| diagnoses were established using two sep! arate diagnostic checklists] one listing DSMIV diagnostic criteria for schizophrenia and the other ICD09 diagnostic criteria[ Next to each criterion or sub!criterion\ clinicians circled one of three possible response options] "0#\ if the symptom was not present^ "4#\ if the symptom was present and "8#\ if the clinician was uncertain[ While these check! lists were generated for the purpose of the current study\ criteria were listed using the exact wording from each classi_cation system[
1[2[ Procedure The CIDI interviews were conducted in private[ Written informed consent was obtained from all subjects before the interviews began[ Subjects were paid ,19 for their participation[ The two CIDI interviewers were four! year trained psychology graduates who had recently com! pleted a one!week intensive training course in admin! istering the CIDI at the WHO designated training centre in Sydney[ Given the high proportion of questions requir! ing interviewer judgements\ the _rst six interviews were rated by both interviewers[ This allowed rating dis! crepancies to be resolved in the initial stages[ To enable interrater reliability to be calculated across all the inter! views\ a further three interviews were double rated later in the study[ Average CIDI psychosis module interview time was 13 min "S[D[ 01 min#[ The treating clinicians were given both the ICD09 and DSMIV checklists to complete in the days after each CIDI interview had been completed[ Clinicians were not given access to the information obtained from the CIDI interview[ Where two treating clinicians were available "n 32#\ both clinicians completed the checklists[ Where there were discrepancies between the checklists\ the clin! icians met\ resolved the discrepancies and completed a third\ consensus checklist which was used as the validity standard[ This procedure generated a more reliable val! idity standard while also enabling the reliability of the clinicians| diagnoses to be assessed[ Eight clinicians par! ticipated "senior psychiatry residents\ consultant psy! chiatrists\ and clinical psychologists#[ All clinicians had extensive experience in the treatment and assessment of psychiatric disorders[ After all data were collected\ the WHO scoring pro! gram for the CIDI was used to generate ICD09 and DSMIV diagnoses[ As the entire CIDI was not admin! istered in the study\ some of the exclusion criteria for schizophrenia "e[g[ the presence of other mental dis! orders# could not be assessed and\ therefore\ were not included in the _nal diagnoses[ Clinician completed checklists were scored according to ICD09 and DSMIV criteria "excluding the criteria not addressed in the CIDI#[ As {uncertain| was not a response option in the CIDI\ all
252
8s were recoded as 0 "not present# in the _nal analysis^ i[e[ the more conservative estimate was used[ 1[3[ Analysis To measure interrater reliability between the clinicians\ Cohen|s kappas "0859# were calculated as a measure of chance!corrected agreement[ Interpretation of kappa was based on Fleiss "0870#[ To measure the concordance between the CIDI and the checklists\ percent agreement and sensitivity and speci_city statistics were calculated[ Statistics were cal! culated at the level of subcriterion\ criterion and diag! nostic level for both ICD09 and DSMIV diagnostic criteria[ Sensitivity measures the proportion of subjects positively diagnosed by the clinicians also positively diag! nosed by the CIDI[ Speci_city measures the proportion of subjects not diagnosed by the clinicians with schizo! phrenia who are also not diagnosed by the CIDI[ While the kappa statistic is often calculated in studies to assess validity\ several factors make sensitivity and speci_city more appropriate statistical measures for this design "see Maclure and Willett\ 0876 for a review#[ Firstly\ kappa assumes that the same instrument is being employed by the raters on both occasions[ In other words\ kappa treats the diagnostic standard "clinicians| diagnosis# as identical to the instrument being validated "CIDI#[ This makes kappa more a measure of reliability rather than validity "Anthony et al[\ 0874^ Helzer et al[\ 0874^ Robins\ 0874#[ Secondly\ kappa is sensitive to sample characteristics such as prevalence rates[ This limits the generalizability of the results of the study and renders comparisons between studies uninformative and misleading "Fleiss\ 0870#[ Thus\ kappa was not used as a statistical measure of validity in this study[
2[ Results 2[0[ CIDI interrater reliability The nine double rated CIDI interviews were inves! tigated for any discrepancies[ For individual criteria across the interviews\ only two had more than one dis! crepancy[ These were catatonic behaviour and delusions\ both coded using interviewer judgements[ 2[1[ Clinician checklist interrater reliability To assess the integrity of the clinicians| diagnoses\ kappas were calculated at the sub!criterion\ criterion and diagnostic level for both ICD09 and DSMIV diagnostic criteria[ The agreement between the clinicians ranged from good to excellent "see Table 0#\ with most kappas above 9[64[ Kappa for agreement between the clinicians for the diagnosis of schizophrenia was 9[71 for DSMIV
253
L[ Cooper et al[:Journal of Psychiatric Research 21 "0887# 250Ð257 Table 0 Interrater reliability between clinicians on the DSMIV and ICD09 checklists for the diagnosis of schizophrenia DSMIV Criteria
Kappa
ICD!09 General criteria
Kappa
184[XX Diagnosis of schizophrenia Criterion A A0 Delusions Bizarre delusions A1 Hallucinations A2 Disorganised speech A3 Catatonia A4 Negative symptoms Criterion B ] social:occupational Criterion C ] duration Criterion E ] substance:medical exclusions
9[71 9[73 9[74 9[70 9[76 9[75 9[64 9[85 9[62 9[89 9[64
F19[X Diagnosis of schizophrenia Criterion G0 0A Thought echo 0B Control delusions 0C Hallucinatory voices 0D Persistent delusions 1A Persistent hallucinations 1B Neologisms 1C Catatonia 1D Negative symptoms Criterion G1[1 ] substance:medical exclusions
9[60 9[70 9[70 9[79 9[73 9[67 9[80 9[74 9[70 9[58 9[69
and 9[60 for ICD09[ These results indicate the validity standard to be reliable[ 2[2[ Clinician checklists vs CIDI Amongst the 62 subjects\ the CIDI made 6 DSMIV diagnoses of schizophrenia and 03 ICD09 diagnoses of schizophrenia[ This was considerably fewer than the clin! icians who made 39 DSMIV diagnoses and 23 ICD09 diagnoses of schizophrenia[ Table 1 presents the sen! sitivity and speci_city scores and percent agreement for the CIDI at the subcriterion\ criterion and diagnostic level for ICD09 and DSMIV[ Firstly\ for DSMIV criteria\ the performance of the CIDI was mixed[ Percent agree! ment was 49[6 for the diagnosis of schizophrenia\ and ranged between 32[6 and 66[4 at the criterion level[ Sensitivity ranged from very good "9[82 bizarre delusions#\ to very poor "9[07 negative symptoms#[ Over! all\ speci_city scores were better\ but still varied "range
9[27Ð9[84#[ In general\ the sensitivity of the CIDI was particularly poor for the sub criteria scored at the end of the interview using interviewer judgements] disorganised speech\ negative symptoms and catatonia\ resulting in poor sensitivity for General Criterion A[ The sensitivity for the diagnosis of schizophrenia was very low "9[04#\ with high speci_city "9[86#[ For ICD09 criteria\ the pattern of results is repeated[ Agreement was 46[7) for the diagnosis of schizophrenia and ranged between 31[2 and 66[4) at the criterion level[ Sensitivity varied considerably from low "9[08 neol! ogisms# to high "9[89 persistent delusions#[ Speci_city also varied but was higher "range 9[28Ð9[84#[ Again\ the sensitivity of the CIDI was consistently poor for the sub criteria scored at the end of the interview using the interviewer judgements\ neologisms and negative symp! toms\ resulting in poor sensitivity for General Criterion 0[ The sensitivity for the diagnosis of schizophrenia was again low "9[15# and speci_city\ high "9[75#[
Table 1 CIDI vs clinician checklists ] percentage agreement\ sensitivity and speci_city for DSMIV and ICD09 at the diagnostic\ criterion and sub criterion level
DSMIV criteria 184[XX Schizophrenia Criterion A A0 Delusions Bizarre delusions A1 Hallucinations A2 Disorganised speech A3 Catatonia A4 Negative symptoms Criterion B ] social:occupational Criterion C ] duration Criterion E ] substance:medical exclusions
) agreement Sensitivity Speci_city ICD!09 criteria 49[6 41[0 66[4 62[1 56[5 53[7 34[0 52[3 55[1 32[6 55[1
9[04 9[26 9[82 9[66 9[75 9[10 9[52 9[07 9[57 9[19 9[57
9[86 9[75 9[44 9[56 9[49 9[84 9[27 9[82 9[47 9[84 9[52
F19[X Schizophrenia Criterion G0 0A Thought echo 0B Control delusions 0C Hallucinatory voices 0D Persistent delusions 1A Persistent hallucinations 1B Neologisms 1C Catatonia 1D Negative symptoms Criterion G1[1 ] substance:medical exclusions
) agreement Sensitivity Speci_city 46[7 41[0 42[4 69[3 69[3 66[4 48[1 55[1 31[2 52[3 55[1
9[15 9[24 9[71 9[56 9[59 9[89 9[48 9[08 9[56 9[19 9[60
9[75 9[73 9[37 9[61 9[64 9[52 9[59 9[82 9[28 9[84 9[59
L[ Cooper et al[:Journal of Psychiatric Research 21 "0887# 250Ð257
3[ Discussion This study presents data on the validity of the psychosis module of the interviewer administered CIDI!Auto "1[0#[ Validity was assessed by examining the concordance between the CIDI!Auto as administered by trained\ lay interviewers and diagnostic checklists completed by clin! icians[ Unlike previous validity studies on the CIDI "e[g[ Janca et al[\ 0881^ Kovess et al[\ 0881#\ the validity stan! dard of this study was not contaminated by allowing the clinicians access to the information obtained in the CIDI interview[ Further\ as can be seen from the good inter! rater reliability between the clinicians\ the structured for! mat of the checklists provided a reliable validity standard[ Overall\ results from the study suggest that compared to clinicians| diagnoses\ the psychosis module of the CIDI has good speci_city\ but poor sensitivity[ For DSMIV criteria the sensitivity and speci_city for the diagnosis of schizophrenia was 9[04 and 9[86 respectively and for ICD09 criteria\ the sensitivity and speci_city was 9[15 and 9[75 respectively[ This suggests that while the CIDI was accurate in the positive diagnoses it made\ it had a high {miss| rate\ failing to detect many of the positive diagnoses made by the clinicians[ Compared to the clin! icians\ the CIDI detected only 07) of DSMIV diagnoses of schizophrenia\ and 30) of ICD09 diagnoses of schizo! phrenia resulting in a considerable underestimation of the {true| prevalence rate in this sample[ These _ndings are in accordance with earlier studies investigating the validity of the psychosis module of the DIS[ Results from these studies show the psychosis mod! ule to considerably underestimate the prevalence of schizophrenia in both general and clinical populations "Wittchen et al[\ 0874#\ for patients in both active and non!active phases of the disorder "Semler et al[\ 0876^ Spengler and Wittchen\ 0877^ Wittchen et al[\ 0874#\ and for lifetime and current diagnosis "Spengler and Wittchen\ 0877#[ In the study by Wittchen et al[ "0874#\ the psychosis module had the poorest concordance with structured clinical diagnoses "LEAD standard# out of all the DIS modules[ Similarly\ in a study which examined the validity of the DIS when administered by trained lay interviewers\ the DIS failed to detect one third of subjects with a well documented history of the disorder "Pulver and Carpenter\ 0877#[ While the considerable time delay of six years between the clinicians| diagnoses and DIS interview necessitates caution in interpreting this par! ticular result\ it is consistent with the general _nding that the psychosis module considerably underestimates prevalence of the disorder[ Finally\ in the more recent study by Rosenman et al[ "0886# using CIDI!Auto 0[0\ sensitivity for the ICD09 diagnosis of schizophrenia and related disorders was 9[35 and speci_city 9[75[ The lower sensitivities obtained in the present study "9[15# could re~ect the fact that the self!administered version of CIDI! Auto does not include the interviewer!rated questions at
254
the end of the interview which generally performed poorly "e[g[ neologisms\ negative symptoms# and this utilised a less stringent validity standard[ While variations in sample characteristics and method! ologies between these previous studies and the current study necessitate caution when making comparisons\ the similar pattern in results suggests particular problems with the validity of the psychosis module[ An advantage of the current study was that the validity standard enabled a more sensitive analysis of the performance of the CIDI at the criterion and sub criterion levels[ The performance of the CIDI varied considerably between individual criteria[ For ICD09 criteria\ sensitivities ranged between 9[08 "neologisms# and 9[89 "persistent delusions#\ and for DSMIV criteria sensitivities ranged between 9[07 "negative symptoms# and 9[82 "bizarre delusions#[ For both diagnostic systems\ the sensitivities for 5 out of the 09 criteria were 9[59 or above[ This suggests that the low sensitivity obtained for the diagnosis of schizophrenia may be attributable to the poor per! formance of the CIDI on a small number of criteria[ For DSMIV\ the poorly performing criteria were dis! organised speech\ duration greater than six months\ and negative symptoms "sensitivity 9[10^ 9[19 and 9[07 respec! tively#\ and for ICD09 they were neologisms and negative symptoms "sensitivity 9[08 and 9[19 respectively#[ Apart from duration for DSMIV\ all these poorly performing criteria are scored from the _nal 00 questions rated by the interviewer at the end of the interview[ These require the interviewer to judge\ for example\ if the subject is able to {persist in goal directed activities|\ or whether they exhibit features of {~at a}ect|\ or {thought disorder[| Two explanations could account for the low concordance between the CIDI interviewers and clinicians for these items[ Firstly\ it is possible that only administering the psychosis module of the CIDI did not allow su.cient time for interviewers to observe the behaviours to be rated and hence\ did not allow the interviewers to make accurate judgements[ Secondly\ the sensitivity for these items may have been poor because they were assessed using the judgements of lay interviewers[ While other criteria are assessed in the module on the basis of subject self!report\ these criteria are assessed entirely on the basis of interviewer judgement[ Given the complex clinical nat! ure of these criteria\ it is probable that interviewers who do not have a clinical background do not possess the necessary clinical skills to make these judgements[ In support of this second explanation\ the psychosis module is the only module in the CIDI for which adequate validity has not yet been established "see Wittchen\ 0883# and it is the only module which depends on interviewer judgement to establish diagnoses[ For all other modules\ diagnoses are made solely on the basis of subject self!report[ A problem unique to the psychotic disorders\ however\ is that diagnostic criteria relating to facial expression and speech patterns can only be assessed
255
L[ Cooper et al[:Journal of Psychiatric Research 21 "0887# 250Ð257
by an outside observer[ In light of this restriction\ one means of improving the validity of this module would be to extend the current interviewer training program to focus speci_cally on the identi_cation and assessment of these criteria[ The better sensitivities obtained for the criteria relating to the more obvious symptoms of delusions and hallucinations would suggest that lay inter! viewers are able to distinguish between {normal| and clini! cally signi_cant experiences when provided with self! reported symptoms[ Alternatively\ how these criteria are currently described and operationalised in the CIDI could be improved[ Sensitivities for these items may be low simply because they have poor construct validity[ In other words\ interviewers are not coding these items incorrectly because they lack the necessary skills\ but because they are poorly conceptualised and described by the CIDI text[ A way to investigate this hypothesis would be to have trained\ experienced clinicians administer these items[ If poor sensitivities were still obtained for these items\ this would suggest problems in their oper! ationalisation as opposed to problems with the judge! ments of lay interviewers[ Other factors\ however\ are also likely to be con! tributing to the overall poor validity of this module[ While the tendency for structured diagnostic interviews in general to underdiagnose compared to clinicians has been well documented in the literature "e[g[ Fennig et al[\ 0883^ McGorry et al[\ 0889^ Wittchen et al[\ 0874#\ fea! tures of psychosis are considered to make its assessment via structured interviews particularly tenuous[ Notably\ the episodic nature of the illness is believed to make obtaining an accurate assessment from a single interview very di.cult\ with results ~uctuating depending upon the illness {phase| in which the interview takes place "Wittchen et al[\ 0878#[ In the non!active phase of the disorder\ fear of social stigmatisation associated with psy! chotic disorders is believed to lead subjects to either deny\ or minimise experiencing symptoms "Anthony et al[\ 0874^ Erdman et al[\ 0876^ Pulver and Carpenter\ 0877#[ This is evident in the particularly poor recall of symptoms shown by subjects with no ~orid symptoms at the time of interview "Wittchen et al[\ 0874#[ Then\ in the {active| phase of the disorder\ features of psychosis such as de_! cits in a}ect\ insight and reasoning interfere with the subject|s ability to give coherent and accurate responses "Farmer et al[\ 0876^ McGorry et al[\ 0878^ Pulver and Carpenter\ 0887#[ In support of this\ it has been shown that the accuracy of CIDI diagnoses notably decreases for cases in an active phase of schizophrenia "Semler et al[\ 0876#[ This is likely to have been particularly prob! lematic in the current study\ given that the sample came predominantly from an acutely ill population[ These di.culties associated with psychosis have often been raised as arguments against the use of structured interviews such as the CIDI for assessment of more
chronic disorders such as schizophrenia[ The problem with these self!report instruments is that their reliability and validity is ultimately dependent on the lucidity and compliance of the subject[ In the case of psychoses\ however\ this cannot be depended upon and testi_es to the need to include sources of information other than subject self!report to establish accurate diagnoses[ For instance\ information collected by the CIDI could be supplemented with information obtained from medical records or interviews with close relatives of the subject[ More generally\ however\ it should be noted that the endeavour to achieve perfect agreement with diagnoses made by clinicians is unrealistic[ Firstly clinicians are able to draw upon multiple sources of information and their own clinical experience in reaching diagnoses[ As well\ structured diagnostic interviews are more stringent in their interpretation and application of the diagnostic cri! teria meaning that compared to clinicians\ they will always tend to underdiagnose {true| prevalence rates[ Secondly\ the less than perfect agreement found between clinicians in the diagnosis of schizophrenia creates a {ceil! ing|] the validity of the instrument can only be as good as that of the diagnostic standard it is compared with[ Kappas of between 9[58 and 9[60 for example\ are gen! erally reported for the diagnosis of schizophrenia by two clinicians "Robins\ 0874^ Winokur et al[\ 0877#[ In this study\ while the structure of the checklists ensured the clinicians applied the criteria\ kappas for the diagnosis of schizophrenia were still only 9[71 for ICD09 criteria\ and 9[60 for DSMIV[ Hence\ until a gold standard in psychiatry is obtained\ the validity of instruments such as the CIDI will always be less than perfect[ Finally\ it should also be noted that while the CIDI was designed primarily as an epidemiological instrument for use in general populations\ as with most previous validity studies on the psychosis module\ the current study was conducted within a psychiatric setting[ Given the nature of psychosis\ it is likely that the way in which this module in particular performs in a clinical population will be di}erent to the way in which it performs in the general population[ Indeed\ a recent study by Kendler et al[ "0885# using a modi_ed University of Michigan ver! sion of the CIDI "UM!CIDI# to assess lifetime prevalence rates of disorders in the general population showed just this[ While the lifetime prevalence rate for schizo! phrenia:schizophreniform disorder was estimated to be 0[2) by the CIDI\ the equivalent prevalence rate based on clinicians| diagnoses was 9[1)[ Only 09) of the posi! tive diagnoses made by the CIDI were con_rmed by the clinicians[ In other words\ the psychosis module when used in the community considerably overdiagnosed the prevalence of schizophrenia and related disorders\ thereby reversing the trend found in clinical populations[ One explanation given for the high false positives found in the Kendler et al[ "0885# study was that the threshold for judging if a symptom is {psychotic| is too low in the
L[ Cooper et al[:Journal of Psychiatric Research 21 "0887# 250Ð257
CIDI[ This would indicate a problem with the plausibility judgements made by interviewers to assess the clinical signi_cance of {unusual| experiences[ The good sensitivity and speci_city found for these items in the current study\ however\ suggests that these judgements are only di.cult to make in general populations in which the symptoms are likely to be less obvious[ In conclusion\ the following directions for future research are recommended in order to improve the poor validity of the psychosis module[ Firstly\ the module|s current reliance on interviewer judgement at the end of the interview\ particularly in relation to the assessment of negative symptoms\ needs to be addressed[ Secondly\ using other sources of information to supplement the CIDI in establishing diagnoses\ particularly in a psy! chiatric setting\ needs to be considered[ This would also help to identify additional reasons for the module|s cur! rent high false negative rate[ Finally\ the way in which the module performs in less severely disturbed populations needs to be determined\ particularly in relation to its ability to discriminate threshold or subclinical cases from {true| cases[ This is particularly crucial given the CIDI|s primary role as an epidemiological instrument[
Acknowledgements This study was supported by a grant from the Com! monwealth of Australia\ Research and Development Grants Advisory Committee[
References American Psychiatric Association[ Diagnostic and statistical manual of mental disorders[ 3th ed[ Washington\ DC] American Psychiatric Association\ 0883[ Andrews G\ Peters L\ Guzman A\ Bird K[ A comparison of two struc! tured diagnostic interviews] CIDI and SCAN[ Australian and New Zealand Journal of Psychiatry 0884^18]013Ð21[ Anthony JC\ Folstein M\ Romanowski AJ\ von Kor} MR\ Nestadt GR\ Chadal R\ Merchant A\ Brown H\ Shapiro S\ Kramer M\ Gruenberg EM[ Comparison of the lay Diagnostic Interview Sched! ule and a standardized psychiatric diagnosis[ Experiences Eastern Baltimore[ Archives of General Psychiatry 0874^31]556Ð64[ Burnam MA\ Karno M\ Hough RL\ Escobar JI\ Forsyth AB[ The Spanish Diagnostic Interview Schedule[ Reliability and comparison with clinical diagnoses[ Archives of General Psychiatry 0872^39]0078Ð85[ Cohen\ J[ A coe.cient of agreement for nominal scales[ Educational Psychological Measurement 0859^19]26Ð35[ Dohrenwend BP[ {The problem of validity in _eld studies of psycho! logical disorders| revisited[ Psychological Medicine 0889^19]084Ð 197[ Erdman HP\ Klein MH\ Greist JH\ Bass SM\ Bires JK\ Machtinger PE[ A comparison of the Diagnostic Interview Schedule and clinical diagnosis[ American Journal of Psychiatry 0874^033]0366Ð79[ Farmer AE\ Katz R\ McGu.n P\ Bebbington P[ A comparison between the present state examination and the Composite International
256
Diagnostic Interview[ Archives of General Psychiatry 0876^33]0953Ð 8[ Fennig S\ Craig T\ Lavelle J\ Kovasznay B\ Bromet EJ[ Best!estimate versus structured interview!based diagnosis in _rst!admission psy! chosis[ Comprehensive Psychiatry 0883^24]230Ð7[ Fleiss JL[ Statistical methods for rates and proportions[ 1nd ed[ New York] John Wiley and Sons\ 0870[ Helzer JE\ Robins LN\ McEvoy LT[ A comparison of clinical and Diagnostic Interview Schedule diagnoses] physician reexamination of lay!interviewed cases in the general population[ Archives of Gen! eral Psychiatry 0874^31]546Ð55[ Hendricks LE\ Bayton JA\ Collins JL\ Mathura CB\ McMillan SR\ Montgomery TA[ The NIMH Diagnostic Interview Schedule] a test of its validity in a population of black adults[ Journal of National Medical Association 0872^64]556Ð60[ Janca A\ Robins LN\ Cottler LB\ Early TS[ Clinical observation of assessment using the Composite International Diagnostic Interview "CIDI#] an analysis of the CIDI _eld trials*wave II at the St Louis site[ British Journal of Psychiatry 0881^059]704Ð7[ Kendler KS\ Gallagher TJ\ Abelson JM\ Kessler RC[ Lifetime preva! lence\ demographic risk factors\ and diagnostic validity of non! a}ective psychosis as assessed in a US community sample[ Archives of General Psychiatry 0885^42^0911Ð20[ Kovess V\ Sylla O\ Fournier L\ Favigny V[ Why discrepancies exist between structured diagnostic interviews and clinicians| diagnoses[ Social Psychiatry and Psychiatric Epidemiology 0881^16]074Ð80[ Maclure M\ Willett WC[ Misinterpretation and misuse of the kappa statistic[ American Journal of Epidemiology 0876^015]050Ð8[ McGorry PD\ Copolov DL\ Singh BS[ Royal Park Multidiagnostic Instrument for Psychosis] part I[ Rational and review[ Schizo! phrenia Bulletin 0878^05]490Ð04[ McGorry PD\ Singh B\ Connell S\ McKenzie D\ Van Riel R\ Copolov DL[ Diagnostic concordance in functional psychosis revisited] a study of inter!relationships between alternative concepts of psy! chotic disorder[ Psychological medicine 0881^11]256Ð67[ McGorry PD\ Singh B\ Copolov DL\ Kaplan I\ Dossetor CR\ van Riel RJ[ Royal Park Multidiagnostic Instrument for Psychosis II[ Development\ reliability and validity[ Schizophrenia Bulletin 0889^05]406Ð25[ Peters L\ Andrews G[ Procedural validity of the computerized version of the Composite International Diagnostic Interview "CIDI!Auto# in anxiety disorders[ Psychological Medicine 0884^14]0158Ð79[ Peters L\ Clark D\ Carroll F[ Are computerised interviews equivalent to human interviewers< CIDI!Auto vs CIDI in anxiety and depressive disorders[ Psychological Medicine\ in press[ Pulver AE\ Carpenter WT[ Lifetime psychotic symptoms assessed with the DIS[ Schizophrenia Bulletin 0872^8]266Ð71[ Pulver AE\ Carpenter WT[ Accuracy of the diagnoses of a}ective dis! orders and schizophrenia in public hospitals[ American Journal of Psychiatry 0877^034]107Ð19[ Robins LN[ Re~ections on testing the validity of psychiatric interviews[ Archives of General Psychiatry 0874^31]807Ð13[ Robins LN\ Helzer JE\ Croughan J\ Ratcli} KS[ National Institute of Mental Health Diagnostic Interview Schedule[ Archives of General Psychiatry 0870^27]270Ð8[ Robins LN\ Wing J\ Wittchen H!U\ Helzer JE\ Babor TF\ Burke J\ Farmer A\ Jablenski A\ Pickens R\ Regier D\ Sartorius N\ Towle LH[ The Composite International Diagnostic Interview[ Archives of General Psychiatry 0877^34]0958Ð66[ Rosenman SJ\ Korten AE\ Levings CT[\ Computerised diagnosis in acute psychiatry] validity of CIDI!Auto against routine clinical diagnoses[ Journal of Psychiatry Research 0886^20]470Ð81[ Semler G\ Wittchen H!U\ Joschke K\ Zaudig M\ von Geiso T\ Kaiser S\ von Cranach M\ P_ster H[ Test!retest reliability of a standardized psychiatric interview "DIS:CIDI#[ European Archives of Psychiatry and Neurological Sciences 0876^125]103Ð11[ Spengler PA\ Wittchen H!U[ Procedural validity of standardized symp!
257
L[ Cooper et al[:Journal of Psychiatric Research 21 "0887# 250Ð257
tom questions for the assessment of psychotic symptoms] A com! parison of the DIS with two clinical methods[ Comprehensive Psy! chiatry 0877^18]298Ð11[ Spitzer RL[ Psychiatric diagnosis] are clinicians still necessary< Com! prehensive Psychiatry 0872^13]288Ð300[ Spitzer RL\ Williams JBW[ Procedural validity] validity of the diag! nostic process[ In Kaplan HI\ Freedman AM\ Sadock BJ\ editors[ Comprehensive Textbook of Psychiatry 2rd ed[ Baltimore] Williams and Wilkins Company\ 0879[ Winokur G\ Zimmerman M\ Cadoret R[ Cause the bible tells me so[ Archives of General Psychiatry 0877^34]572Ð3[ Wittchen\ H[ [ Reliability and validity studies of the WHO!Composite International Diagnostic Interview "CIDI#] a critical review[ Journal of Psychiatric Research 0883^17]46Ð3[ Wittchen H!U\ Semler G\ von Zerssen D[ A comparison of two diag! nostic methods[ Clinical ICD diagnosis vs DSM!III and RDC using the DIS "version II#[ Archives of General Psychiatry 0874^31]566Ð 73[
Wittchen H!U\ Burke JD\ Semler G\ P_ster H\ von Cranach M\ Zaudig M[ Recall and dating of psychiatric symptoms] test!retest reliability of time!related symptom questions in a standardized psychiatric interview[ Archives of General Psychiatry 0878^35]326Ð32[ Wittchen H!U\ Robins LN\ Cottler LB\ Sartorius N\ Burke JD\ Regier D[ Participants in the multicentre WHO:ADAMHA _eld trials[ Cross!cultural feasibility\ reliability and sources of variance of the Composite International Diagnostic Interview "CIDI#[ British Jour! nal of Psychiatry 0880^048]534Ð42[ World Health Organization[ The ICD09 classi_cation of mental and behavioral disorders] diagnostic criteria for research[ Geneva\ WHO\ 0882[ World Health Organization[ CIDI!Auto "1[0# administrator|s guide and reference[ Sydney] Fast Books\ 0886[ Zimmerman M[ Why are we rushing to publish DSM!IV[ Archives of General Psychiatry 0877^34]0024Ð7[