Validation of an algorithm for the diagnosis of Restless Legs Syndrome: The Restless Legs Syndrome-Diagnostic Index (RLS-DI)

Sleep Medicine 10 (2009) 515–523 www.elsevier.com/locate/sleep

Original Article

Validation of an algorithm for the diagnosis of Restless Legs Syndrome: The Restless Legs Syndrome-Diagnostic Index (RLS-DI) Heike Benesˇ a,*, Ralf Kohnen b a

Somni Bene Institute for Medical Research and Sleep Medicine Schwerin and Neurology Department, University of Rostock, Germany b IMEREM Institute for Medical Research Management and Biometrics, Nuremberg, and Psychology Department, University of Erlangen-Nuremberg, Germany Received 19 June 2007; received in revised form 4 June 2008; accepted 5 June 2008 Available online 26 September 2008

Abstract Background: There is a need for structured methods to improve sensitivity and specificity of diagnostic decision making in Restless Legs Syndrome (RLS). We present the RLS-Diagnostic Index (RLS-DI), a diagnostic algorithm which combines essential and supportive diagnostic criteria from patient interviews, polysomnography and neurological examination in an adaptive procedure. Method: The RLS-DI consists of 10 items which are related to the essential diagnostic criteria established by the International RLS Study Group (five items) as well as their supportive criteria (3 items) and features associated with RLS (2 items). Items have to be completed using three categories per item that address frequency of occurrence of symptoms or certainty of presence or absence of other diagnostic information. Negative weights were given when the clinically most relevant items were not present. The RLS-DI was administered in a telephone interview to 179 patients (86 with RLS, 93 with other sleep disorders) of the 21 month cohort of one sleep center in Germany. Results: With receiver-operating characteristics, a cut-off of P11 points on a scale ranging from 22 (no RLS) to 20 (definite RLS) was identified by comparing the RLS-DI total score to the diagnosis of two independent sleep experts. Sensitivity was 93.0%, specificity was 98.9%, and 96.1% of the patients could be correctly diagnosed. Specificity was higher in items related to supportive or associated diagnostic information (95.7%) than in those related to the essential diagnostic criteria (81.7%). Patients with RLS scored a higher RLS-DI than those with primary insomnia or other neurological or psychiatric disorders (p < .001). Conclusion: The RLS-DI demonstrated the ability to validly diagnose an actual and persistently present Restless Legs Syndrome in patients of a sleep lab population and to exclude those patients whose sleep disturbances have other causes. Ó 2008 Elsevier B.V. All rights reserved. Keywords: Restless Legs Syndrome; RLS; Urge to move; Diagnosis; Structured interview; Validation

1. Introduction Restless Legs Syndrome (RLS) is defined as a sensorimotor disorder. Its key symptom is an urge to move or a * Corresponding author. Address: Somni bene GmbH, Arsenalstrasse 10, 19053 Schwerin, Germany. Tel.: +49 385 20 79 80; fax: +49 385 20 79 813. E-mail address: [email protected] (H. Benesˇ).

1389-9457/$ - see front matter Ó 2008 Elsevier B.V. All rights reserved. doi:10.1016/j.sleep.2008.06.006

focal akathisia focused on the legs (and arms in some patients). This focal akathisia is often associated with dysesthesias, i.e., unpleasant abnormal sensations occurring without any apparent sensory stimulation. The urge to move and any accompanying dysesthesias must be engendered by rest, relieved by movement and experienced worse in the evening or night with some relief in the morning [1]. RLS is a clinical diagnosis based primarily on self-reports. The International RLS Study

516

H. Benesˇ, R. Kohnen / Sleep Medicine 10 (2009) 515–523

Group has published [2] and amended [3] diagnostic criteria for improving decision making concerning the presence or absence of RLS based on this subjective data (see Web Appendix). However, specificity of the criteria is not perfect, and ‘‘mimics” (false positive diagnoses) of RLS have been reported in more than 10% of the investigated populations [4–6], i.e., other conditions such as nocturnal cramps or complaints of neuropathy can fulfill the essential diagnostic criteria indicating the need for a more differentiated diagnosis [4,7,8]. On the other hand, RLS is thought to be under diagnosed (false negative diagnoses) in routine practice [8–10]. When applying the four essential diagnostic criteria in interview studies conducted by RLS experts, estimates of sensitivity varied between 90% [11] and 97% [12]; in studies with patient completed questionnaires, it was 82% [5]. False negative diagnoses of RLS are frequent if examiners do not understand the criteria correctly, and sensitivity lower than 100% may result from an inadequate diagnostic evaluation. No manual has been published describing how the criteria should be applied. For instance, the time period to be evaluated is not defined. Sensitivity will be different if lifetime or a period of 1 or 4 weeks prior to the assessment is taken into account. Mainly in early and intermittent stages of the disease, symptom-free periods of months or years are common. Sensitivity will also be influenced by the frequency with which RLS symptoms are occurring; frequency may vary between every day and less than once a year. Another weakness of the IRLSSG criteria is the lack of information about how essential and supportive criteria or associated features are related. To improve the validity of RLS diagnosis, structured and standardized methods are needed. The ‘‘Johns Hopkins telephone diagnostic interview for the ‘‘Restless Legs Syndrome”, [11,12] is designed as a structured diagnostic interview to be conducted by RLS experts: the diagnosis is established by assessing six features of RLS which cover the four essential criteria. The main outcome of the interview is a diagnosis which is classified according to its certainty (definite, probable, possible, no diagnosis of RLS). Two studies showed sensitivities of 90% [11] and 97% [12] and specificities of 90% and 92%, respectively. Here we report on the validation of a different diagnostic approach, the RLS-Diagnostic Index (RLS-DI), a standardized diagnostic tool to be completed by experts or non-experts. It is a diagnostic algorithm which (a) identifies patients with a definite diagnosis of RLS, (b) excludes the diagnosis of RLS in other patients, and (c) sorts out a subgroup of patients in whom RLS is possible and cannot be safely excluded or confirmed at the time of the interview without further diagnostic information. The concepts behind the RLS-DI and the results of a first validation study are reported.

2. Methods 2.1. Development of the RLS-DI The RLS-DI consists of 10 items (Appendix 1). All essential criteria from the IRLSSG are included with criterion 1, ‘‘urge to move usually accompanied or caused by uncomfortable or unpleasant sensations in the legs,” split into two items. This split was considered necessary to avoid false positive diagnoses when patients report dysesthesias but without the urge to move [13]. The criteria classified as ‘‘supportive” for RLS by the IRLSSG are integrated into three further items. From the ‘‘associated features” we selected sleep disturbance and findings of a neurological assessment for our item pool, but not information on the course of the disease which we considered not specific enough for our diagnostic purposes since it is based on long-term observation. The diagnostic validity of all criteria is extensively discussed in publications on the IRLSSG diagnostic criteria [3,13]. In the development of the RLS-DI four major decisions have been taken (for considerations behind our decisions, see discussion Section 4): 1. The presence or absence of individual diagnostic information is weighted. 2. We do not require ‘‘yes–no” decisions but quantify each item by providing three alternative frequencies of occurrence or levels or certainty regarding other diagnostic information. 3. A time frame of 7 days was chosen for this first version of the RLS-DI. It is intended to be used in subjects with persistent and actually present symptoms asking for an immediate diagnostic decision for their acute sleep problems or unpleasant sensations in their legs (or arms). 4. The RLS-DI can be used within an adaptive procedure and collects diagnostic information in defined steps depending on the certainty of the diagnostic decision during the diagnostic process. The current version of the RLS-DI is shown in Appendix 1. It is divided into two parts: the essential criteria are questioned in items 1–5 (Part A) and the nonessential diagnostic information is covered by items 6– 10 (Part B). 2.2. Validation study A first validation study was designed and conducted with the cohort of all patients attending our sleep laboratory for evaluation of sleep-related disorders between 01 January 2003 and 30 September 2004. Patients were excluded who were referred with a primary diagnosis of epilepsy or sleep apnea syndrome. A homogenous

H. Benesˇ, R. Kohnen / Sleep Medicine 10 (2009) 515–523

sleep laboratory population has an advantage in that almost all essential, associated and supportive diagnostic information is available for most of the patients. A written invitation was mailed to the patients asking them to participate in a telephone interview lasting about an hour to answer questions on the severity of their sleep and movement disturbances and quality of life. In order to improve the quality of the interview data, patients who responded positively were mailed a full copy of the interview beforehand. Telephone interviews were conducted between November 2004 and January 2005 by two extensively trained interviewers who were not RLS experts. During the fully structured telephone interview, patient characteristics, information on type and duration of symptoms and current treatment were documented. Patients were asked to respond to all questions from three scales which assess severity of RLS symptoms (IRLSSG severity rating scale [14], RLS-6 scales [15], John Hopkins RLS Severity Scale [16]), the WHO-5 scales [17] as screeners for depressive symptoms, a sleep questionnaire (SF-B) [18], and the Short-Form Health Questionnaire in its 12-item version to determine quality of life [19]. We will report data on these scales in a separate publication [20]. Finally, the RLS-DI was completed. Items 8 (response to dopaminergic drugs), 9 (polysomnography or actigraphy results) and 10 (neurological examination) were answered from the patients’ sleep lab files. If one of these features could not be assessed at the time of the diagnostic evaluation they were scored as zero. The majority of patients had started treatment for RLS or other disorders after their visit to the sleep lab. Therefore, patients under treatment at the time of the interview were asked to answer the questions in a twofold manner: for the last week and for the time before commencement of treatment of their symptoms. For the evaluation of the RLS-DI, the retrospective data were used. Instead of the term ‘‘RLS symptoms” or something similar in some of the scales, the wordings ‘‘urge to move your legs (and arms)” and, if appropriate, ‘‘with unpleasant sensations” were used in the interview. Two senior physicians with expertise in diagnosis and treatment of RLS and other sleep disorders provided the RLS expert diagnoses for each patient, one of whom had met the patient personally during their stay at the sleep lab and performed all diagnostic assessments. For the purpose of this study, both experts reviewed each patient’s sleep lab file independently and made a primary diagnosis for the patient’s sleep disturbances, but also recorded any sleep-relevant comorbidities as secondary diagnoses. Each expert identified the most bothersome symptom for the patient at the time of their sleep lab examination and specified a primary diagnosis for that symptom. If the primary diagnosis was not identical, the patient’s file was reviewed by both experts again to achieve final agreement on the primary diagno-

517

sis. The large numbers of individual diagnoses were coded according to ICD-10 and summarized as follows: RLS, primary insomnia, other neurological disorder (e.g., polyneuropathy), psychiatric disorder (e.g., depression), other disorders (e.g., orthopedic dysfunction). For the purpose of the RLS-DI validation, two groups of patients were established, those with RLS as primary expert diagnosis and those with another primary diagnosis for the sleep disturbances according to the experts. 2.3. Statistical evaluation Agreement in the diagnosis of sleep disturbances by the two sleep experts was assessed with Cohen’s kappa to determine inter–rater reliability. The distribution of the ratings in the different items of the RLS-DI was analyzed descriptively. For construct validity, we performed a factor analysis including all 10 items with all patients who had a primary or secondary diagnosis of RLS (n = 94), except one patient with no evaluation of a response to dopaminergic medication (score = 0 in item 8). Due to the adaptive nature of the RLS-DI, at minimum we required a homogeneous factor structure for the essential criteria. In addition, two sum-scores were analyzed, one consisting of the five categories representing the essential criteria (part ‘‘A”) and the other of the five categories representing the supportive and associated features (part ‘‘B”). Receiver-operating characteristic (ROC-) curves were constructed which related pairs of true (sensitivity) and false positives (1-specificity) for each scale point of the RLS-DI to each other point. This made it possible to determine a cut-off for a definite diagnosis of persistent and actually present RLS. The area under this curve (AUC) is a quality criterion which assesses the discriminative power of the RLS-DI as indicated by AUC values far beyond 0.5 (random chance) and close to 1.0 [21]. We calculated sensitivity, specificity, positive and negative predictive values and accuracy of the RLS-DI total score and sub-scores A and B. Criterion validity of the RLS-DI was assessed using the final primary diagnosis of the RLS experts. Discriminant validity was analyzed by comparing the RLS-DI total score of RLS patients to RLS-DI total scores of those with other diagnoses using the Mann–Whitney U-test. Test–retest reliability and inter–rater reliability of the two interviewers could not be determined in this first validation trial as it was thought that repeat measurement would be too demanding on the patients. Cronbach’s alpha was used to determine internal consistency of the 10 items as a reliability estimate for the RLS-DI. Several analyses with weights other than ‘‘ 4” for the ‘‘absent” or ‘‘no” categories of the RLS-DI items were performed, namely ‘‘0” (no weighting of answer categories), ‘‘ 1,” and ‘‘ 2” instead of ‘‘ 4.” Our analyses show that the highest specificity was achieved with the

H. Benesˇ, R. Kohnen / Sleep Medicine 10 (2009) 515–523

518

weight of ‘‘ 4;” sensitivity was not so affected by the different weights if the cut-offs were adjusted according to a different distribution of scores. 3. Results

related to the essential criteria (1–5); the second factor comprised items 6–10 and further explained 18.4% of the variance. The total variance explained by the two factors was 47.7%. The items with the highest factor loadings were items 3 and 5 on the first factor and 10 and 9 on the second factor.

3.1. Patients The sleep lab cohort of all patients requiring evaluation of sleep disturbances by polysomnography in the chosen time period was 265 patients. Of those, 179 responded to the invitation letter and 156 patients could be interviewed; 18 refused and five were not available for the interview. In a second mailing, the 86 patients who did not respond to the first mailing were contacted again. On this occasion, 38 responded, 25 could be interviewed and 13 refused to participate. In total, 179 patients (67.5%) of the cohort participated in this study, 86 with a primary diagnosis of RLS and 93 with other diagnoses (Table 1). Of the latter, 13 patients were suspicious for secondary RLS or had a diagnosis of mild intermittent RLS as comorbidity. The median age of all patients was 58.8 years (range 29–84) and 64.8% were females. Among RLS patients, 88.4% were currently under treatment, while 58.1% of the patients with other diagnoses were receiving disease-specific treatment at the time of the interview. 3.2. Reliability of expert diagnoses

3.4. Definition of a cut-off point for the diagnosis of RLS

The inter–rater reliability between the two experts was j = 0.933 after the first diagnostic work-up. Discrepancies were related to the primary diagnosis in patients with multiple disorders. Complete agreement was achieved after discussion of these eight cases. 3.3. Construct validity 3.3.1. Factor analysis A factor analysis with the ten RLS-DI items using the principal components method delivered two factors according to Kaiser’s criterion: the first factor explaining 29.2% of the total variance included all five items Table 1 Distribution of primary diagnoses of patients from the sleep laboratory cohort Primary Diagnosis

3.3.2. Item convergent validity Correlations between each of the ten individual items and the RLS-DI total score (minus that item) were positive and significant except for item 6, ‘‘sleep disturbances” (r = 0.090). The correlation of the items representing the essential criteria were all larger than r = 0.800, with the highest correlations observed for items 3 (r = 0.892), 5 (r = 0.849), and 1 (r = 0.833). Among the other diagnostic features, item 9 correlated with the total score with r = 0.703, item 8 with r = 0.548, item 10 with r = 0.442 and item 7 with r = 0.332). Item convergent validities above 0.4 are usually acceptable for rating scales [22]. Cronbach’s alpha was 0.68 for the RLS-DI with all 10 items when it was calculated with the sub-population of all patients with a primary or secondary diagnosis of RLS excluding those with no assessment of response to dopaminergic medication; it was 0.89 with all patients. Values of 0.7 or higher indicate adequate internal consistency [23].

Secondary diagnosis of RLS

RLS

86 (48.0)

Not applicable

Other diagnoses Insomnia Neurological disorder Psychiatric disorder Other disordera

93 (52.0) 69 (38.5) 13 (7.3) 9 (5.0) 2 (1.1)

15 5 3 5 2

(8.4) (2.8) (1.7) (2.8) (1.1)

Remarks: The table shows number of patients and percent of total (in parentheses). a Patients with orthopaedic disorders (arthrosis).

Fig. 1 shows the ROC curve for the RLS-DI total score. The cut-off in ROC curves is usually defined as the longest perpendicular from a data point (RLS-DI score) to the diagonal in the graph. With this principle, the cut-off was determined as P11 for the RLS-DI total score which was used for the further analysis of the RLS-DI. The ROC analysis showed a very favorable ratio between high sensitivity and high specificity: the area under the curve was 98.7% (maximum: 100%) for the RLS-DI total score. Thus, the ROC analysis supports the discriminative power of the RLS-DI to separate two groups: those with RLS and those whose sleep problems had other primary causes at the time of the sleep lab investigation [21]. 3.5. Validity analyses 3.5.1. Criterion validity The distribution of RLS-DI total scores for all patients is displayed in the Web Appendix. It demonstrates clear discrimination between RLS-patients who have high RLS-DI total scores and patients with other diagnoses who, with one exception, had RLS-DI total scores below the cut-off of 11 points.

H. Benesˇ, R. Kohnen / Sleep Medicine 10 (2009) 515–523

519

Table 3 Descriptive statistics of the RLS-DI in different diagnostic groups Diagnosis

N

Mean ± SD

Median

RLS Primary insomnia Neurological disorders Psychiatric disorders Other diagnoses

86 69 13 9 2

+15.9 ± 4.5 10.5 ± 7.1 2.3 ± 10.2 +0.4 ± 11.7 +1.5 ± 5.0

+17 14 0 +5 +1.5

Range 11 ± 20 19 ± 10 22 ± 8 14 ± 15 2±5

Remarks: Range of the RLS-DI total score: 22 (no RLS) to +20 (definite RLS). N, number of patients; SD, standard deviation.

Fig. 1. ROC analysis of the RLS-DI total score. Remarks: AUC, area under the curve. The graph depicts the total RLS-DI (primary diagnosis: RLS vs. other diagnoses). Sensitivity (vertical axis) is plotted against false positives (horizontal axis). Positive test results are defined as scores ‘‘1–4 days” and ‘‘5–7 days” (items 1–5) and ‘‘definitely present” or ‘‘uncertainly present” (items 6–9). For item 10, a positive test result was defined as ‘‘other diagnosis questionably present” or ‘‘not present.”

To assess criterion validity, the RLS-DI total score (P11 suggesting a diagnosis of RLS) was compared to the common diagnosis of the two experts (Table 2). The two sub-scores were analyzed in the same way. Sensitivity of the RLS-DI total score is high but not perfect. Most importantly, the specificity of the method is even higher. In total, 7 patients were falsely classified resulting in an accuracy of 96.1% correct diagnoses. There was one false positive RLS diagnosis (RLSDI = 15) in a patient with adjustment disorder, primary insomnia and abuse of hypnotics in whom an intermittent RLS was suspected as comorbidity. Six other patients with a primary diagnosis of RLS at the time of sleep lab assessment were not identified correctly by the RLS-DI (false negative diagnoses). In one patient with a secondary RLS due to iron deficiency, RLS symptoms had disappeared after uterine resection. Four of these patients suspicious for a periodic limb movement disorder were untreated at the time of the interview; one other used levodopa on demand. In addition, in one RLS patient with concomitant depressive disorder and a RLS-DI total score of 10, no L-

DOPA test could be performed due to concomitant medication. One patient suffered from an anamnestic syndrome which caused difficulties to remember the RLS symptoms correctly (RLS-DI = 7). When both sub-scores of the RLS-DI were compared with the total score, the most striking observation is a comparably low specificity of the five items representing the essential criteria (Part A, items 1–5: 81.7%). Regarding accuracy, 23 patients were not correctly diagnosed by these items. As a consequence, the high specificity of the RLS-DI total score is mainly due to contributions from the supportive or associated diagnostic features of Part B (items 6–10). The analysis of the ten individual RLS-DI items revealed that both sensitivity and specificity depend on the level of certainty about whether a sign or symptom was present (data on item analysis will be reported in a separate manuscript [24]). Sensitivity is high if a symptom is present on at least 1 of the last 7 days or is at least ‘‘questionably present;” specificity is high if a symptom is present on 5–7 of the last 7 days or is ‘‘definitely” diagnosed. 3.5.2. Discriminant validity The RLS-DI total score of RLS patients was higher than the scores of all other diagnostic groups (p < .001 for all comparisons, Table 3). Patients with primary insomnia had lower scores than patients with a neurological or psychiatric disorder (p < .01). The latter two groups did not differ in their average RLS-DI total score. 4. Discussion The Restless Legs Syndrome Diagnostic Index (RLSDI) aims to improve the validity of the diagnosis of

Table 2 Validity results for the RLS-DI scores in patients with RLS versus patients with other diagnoses RLS-DI scores

Sensitivity

Specificity

Positive predictive value

Negative predictive value

Accuracy

RLS-DI total score P 11 Essential criteria items 1–5 P 4 Additional features: items 6–10 P 4

93.0 94.2 95.3

98.9 81.7 95.7

98.8 82.6 95.3

93.9 93.8 95.7

96.1 87.7 95.5

Remarks: The table presents values in percent (maximum: 100%). The RLS-DI scores were compared to the expert diagnoses ‘‘RLS” versus ‘‘other diagnosis.”

520

H. Benesˇ, R. Kohnen / Sleep Medicine 10 (2009) 515–523

RLS. This diagnostic tool allows us to diagnose RLS in patients with persistent and actually present symptoms (5–7 days during the last 7 days, RLS-DI P 11) but also patients who definitely do not suffer from RLS when the interview is performed (RLS-DI 6 0). Currently, ‘‘Restless Legs Syndrome” is one diagnostic category without any differentiation, based upon the subjective essential diagnostic criteria. Recently, a need to define ‘‘clinically relevant RLS” has been stated [25]. The need to discriminate ‘‘definite, probable, possible, no diagnosis of RLS,” even as a result of a fully structured diagnostic interview [11], reveals a weakness in the disease concept of RLS. There is neither a discrimination of different course forms like ‘‘sporadic, intermittent, persistent RLS” nor are there definitions of severity classes of, e.g., ‘‘minor, moderate, major” RLS, comparable to, e.g., the classification of depressive episodes. Our study describes RLS patients with a definite diagnosis of RLS: they report symptoms for all or most of the days of the interview period (7 days) that fulfill the essential diagnostic criteria and at least one of the supportive or associated diagnostic features of RLS. These patients probably represent a group of patients with ‘‘major RLS” or ‘‘clinically relevant RLS.” We also found a subgroup of patients where the RLS-DI diagnosis could not be substantiated or definitely excluded (RLS-DI between 0 and 11). This subgroup includes patients with less frequently occurring RLS symptoms (1–4 days during the last 7 days or less frequent), with RLS symptoms occurring in the context of other disturbances, mainly psychiatric or psychosomatic disorders, or patients with incomplete RLS-DI data (no test for dopaminergic response). According to our clinical experience and those of others [11], additional diagnostic investigations might become necessary or the symptoms of the patients are mild and tolerable and do not require immediate intervention. Finally, the RLS-DI is a valuable instrument to exclude RLS. The RLS-DI is based on the diagnostic criteria which have been published by the International RLS Study Group (IRLSSG) [3]. Thus, the RLS-DI does not add any new diagnostic criteria nor is its intention to modify the IRLSSG criteria. The only exception is a split of the first essential criterion (urge to move with or without unpleasant sensation) into two items. The innovation proposed by the RLS-DI are three features with impact on the use of the IRLSSG diagnostic criteria and on decision making concerning the diagnosis of RLS in general. (1) The RLS-DI applies the IRLSSG criteria in a standardized manner by providing time frames for the occurrence of essential symptoms or the degree of certainty about the presence of supportive or associated diagnostic items. The time frame for evaluating the essential items is 7 days. The observation

interval was stratified into ‘‘5–7 days,” ‘‘1–4 days” and less than once during a 7-day observation period. Both sensitivity and specificity of the essential signs and symptoms (Part A) are markedly influenced by the frequency of symptoms. It was our intention to identify patients with a ‘‘persistent” form (stable or progressive or episodic with actual frequent symptoms) of RLS which is defined here operationally by a RLS-DI total score of 11 or higher and urge to move present on at least one of 7 days. It is assumed that in this subgroup of RLS patients with acute symptoms the need for therapy might be considered in routine medical practice or they can be included into clinical drug trials using daily medication. With the strict RLSDI definition of at least 11 points, the rate of patients with an RLS diagnosis in the general population will probably be found to be lower than the approximately 3% of ‘‘RLS sufferers” reported in the epidemiological REST studies [9,10] which used the definition ‘‘bothering symptoms occurring at least twice a week.” A RLS-DI total score of 1–10 (‘‘possible” RLS) probably includes another sub-population of patients, e.g., those with ‘‘intermittently” occurring RLS symptoms or with comorbidities which are associated with RLS-like symptoms. (2) The new aspect of the RLS-DI is the combination of the subjective with the more objective diagnostic criteria in one empirically derived score which now either supports the diagnosis of RLS or contributes to its rejection. According to the IRLSSG, the diagnosis of RLS is primarily established by the presence of the essential diagnostic criteria which are evaluated on the basis of patients’ subjective reports of symptoms. Thus, the expert diagnostic clinical interview which intends to capture and evaluate this subjective information is considered the ‘‘gold standard” for diagnosis of RLS [3,26]. In this school of thought, objective information is classified as ‘‘supportive of” or ‘‘associated with” the essential criteria and is defined as secondary ‘‘nonessential” [26] information which is useful primarily for excluding similar looking disorders (mimics of RLS) [4]. Our study strongly confirms the importance of the supportive or associated clinical features mainly for the specificity of the RLS diagnosis, and thus for the exclusion of alternative diagnoses, as far as they were present in the population of this study. However, the most sensitive item of the RLS-DI was ‘‘response to dopaminergic medication” (item 8), with 94.2% of all RLS patients showing a ‘‘definite” response and a further 3.5% showing a possible (‘‘uncertain”) response. A high sensitivity of more than 90% for this variable has also been reported by other authors [27].

H. Benesˇ, R. Kohnen / Sleep Medicine 10 (2009) 515–523

(3) The high specificity of the RLS-DI was also achieved by another new feature: positive and negative weight was given to the presence or absence of symptoms with probably the highest clinical relevance for confirming or excluding the diagnosis of RLS. Currently there is not much discussion about different relevance of the four essential diagnostic criteria [28]. In a recent article, we defined RLS by its cardinal symptom, urge to move, and consequently the diagnosis must be ‘‘no RLS” if no urge to move is reported by a patient [1]. In contrast, Hening and coauthors consider a ‘‘possible RLS” even if patients do not report an urge to move but only unpleasant sensations [11]. The weighting of diagnostic criteria in the RLS-DI by negative scores if they are absent is based upon our clinical experience in diagnosing RLS and other movement or sleep disorders. The weight ‘‘ 4” was chosen to decrease the chance of receiving an RLS diagnosis if no urge to move (item 1) and two of its main determinants (not engendered by rest, no relief by movement) are reported and if a patient does not respond to dopaminergic medication. The high negative weight is also used if neurological examination reveals other potential causes for the symptoms. No weighting was applied to the circadian pattern of symptoms (item 5) since patients with chronic forms frequently have symptoms during the day (96.6% in our RLS patients) and circadian occurrence of symptoms is also frequent among the RLS mimics (e.g., cramps). The influence of this weight of ‘‘ 4” on the high specificity of the RLS-DI total score was demonstrated in separate analyses. Another advantage of the RLS-DI is the possibility to use the tool in an adapative way: The diagnostic process can be stopped if the RLS-DI diagnosis is confirmed (RLS-DI P 11) or excluded (RLS-DI 6 0) based on items 1–8. The adapative use of the RLS-DI will be presented in a separate manuscript. The data of this validation trial substantiates the innovative RLS-DI algorithm for RLS diagnosis, and thereby our selection of items and weights. With the cut-off of at least 11 points we deviate from previous assumptions that the presence of the four essential criteria will be sufficiently sensitive and specific for a valid diagnosis of a persistent and actually present RLS. Instead, the analyses of the five items related to the essential criteria (Part A) indicate that additional information from non-essential, i.e., the supportive or associated, features is required to improve the specificity of the diagnosis in particular [24]. In principle, patients with a family history of RLS or those with PLM will have a higher chance of reaching the cut-off of 11 points. Our data, with only six RLS patients below the cut-off of

521

11, do not allow an estimate of this potential bias. Also, the frequency of symptoms (at least once during the last 7 days) which is used for the assessment of the essential criteria is open for discussion. The RLS-DI in its current version is not able to identify patients who do not suffer from RLS at the time of the diagnostic assessment, e.g., patients with recurrent episodes of RLS followed by long symptom-free intervals. There is an increasing need for structured approaches which provide guidance on how to apply the IRLSSG diagnostic criteria. This is due to increased awareness about RLS following the large increase of research in epidemiology, pathophysiology, diagnosis and therapy options, and also due to the marketing success of pharmaceutical companies launching approved drugs for the treatment of RLS. This disorder is no longer an issue only for RLS specialists but also for general practitioners or those physicians who are faced with treating secondary RLS, for example, caused by iron insufficiency in renal dysfunction. The RLS-DI is a tool for diagnostic decision making, but in its current version it is not suitable for screening of RLS or for epidemiological studies. Questionnaire methods [5] or 1-item screeners [29] are more appropriate for such purposes. Recently a structured interview [11] comparable to those which have been developed in psychiatry (e.g., the WHO Composite International Diagnostic Interview, CIDI [30]) has been published. For all these instruments, training is always required. A training manual for the RLS-DI is available on request. An advantage of the RLS-DI is the time required for completion, which ranges from 1 to 10 min for evaluation of items 1–7 (and 8 in the event that patients are pretreated with dopaminergics). Further health-economic benefit of the RLS-DI can be assumed since it quickly identifies which patients should be transferred to sleep labs or neurological experts, and it helps with treatment decisions. In summary, the RLS-DI demonstrated its usefulness in validly diagnosing patients with a persistent or ‘‘major” RLS and especially excluding patients with sleep disturbances due to other causes which mimic the essential diagnostic criteria of RLS [4]. The conclusions from our validation study are constrained by the fact that we only interviewed patients with sleep disturbances from a specialist sleep center. The poor specificity of the RLS-DI sleep disturbance item 6 in our study is due to using this source of patients because almost all of them suffered from impaired sleep. On the other hand, our cohort of patients is extreme compared to other populations since we included patients sent by physicians to a sleep center because the causes for sleep disorders could not be diagnosed during routine diagnostic work-up. No diagnostic instruments for RLS are currently in place in routine medical care. Another reservation concerns the partly retrospective nature of our data; patients were

H. Benesˇ, R. Kohnen / Sleep Medicine 10 (2009) 515–523

522

asked to remember their condition before treatment had commenced. Furthermore, the completion of the RLSDI and other questionnaires in a telephone interview by trained interviewers may have introduced bias in our findings which could explain the falsely diagnosed patients (3.9%). We were also faced with structural problems of a validation trial that was designed to compare initial diagnoses during routine work-up with retrospective investigations. Two of the RLS-DI items (8 and 10) could not be scored in the majority of patients with diagnoses other than RLS. In the sleep lab’s routine diagnostic work-up, it was considered unethical to apply dopaminergic medication to patients when no positive effect (improvement) was to be expected (item 8) since the patients neither reported RLS symptoms (urge to move) nor showed PLMs during the first polysomnography night. Although a physical examination was performed in all patients, item 10 could not be scored in patients with other primary diagnoses except in 13 patients suspicious for secondary RLS, since it explicitly addresses RLS symptoms and requires that these are present. We have analyzed the effect of our substitution procedure (score = ‘‘0” in both items): none of the patients in the control group with substitution in one or both items would have achieved the cut-off of at least 11 points in the RLS-DI total score if they had scored ‘‘2 = definite” in either item. Nonetheless, further validation of the RLS-DI will become necessary in trials using a different source of patients, more than one RLS-DI examiner per patient, and a prospective design with simultaneous assessment of the subjective and objective diagnostic criteria of the RLS-DI. Financial disclosure There was no funding for this study. Acknowledgements The authors thank Constanze Meissner, M.D. for serving as a sleep expert, Lisa Kaulfuss and Tanja Muchalski for all their contributions to the conduct of the study, and all our patients for spending their time for the interview. We gratefully acknowledge the important and valuable comments of two anonymous reviewers of Sleep Medicine, and we thank Rosemary Bischoff (Berlin, Germany) for her support to improve the language of the manuscript. Appendix A. Supplementary data Supplementary data associated with this article can be found, in the online version, at doi:10.1016/ j.sleep.2008.06.006.

References [1] Benes H, Walters AS, Allen RP, Hening WA, Kohnen R. Definition of restless legs syndrome, how to diagnose it, and how to differentiate it from RLS mimics. Mov Disord 2007;22(18):S401–8. [2] Walters AS. Toward a better definition of the restless legs syndrome. The International Restless Legs Syndrome Study Group. Mov Disord 1995;10(5):634–42. [3] Allen RP, Picchietti D, Hening W, Trenkwalder C, Walters AS, Montplaisir J. Restless legs syndrome: diagnostic criteria, special considerations, and epidemiology – a report from the restless legs syndrome diagnosis and epidemiology workshop at the National Institutes of Health. Sleep Med 2003;4:101–19. [4] Lesage S, Hening WA. The restless legs syndrome and periodic limb movement disorder: a review of management. Semin Neurol 2004;24(3):249–59. [5] Nichols DA, Kushida CA, Allen RP, Grauke JH, Brown JB, Rice ML, et al. Validation of RLS diagnostic questions in a primary care practice. Sleep 2003;26:A346. [6] Tachibana N, Yao M, Okura M, Shimamoto T. Restless Legs Syndrome could be overestimated by questionnaire based on standard diagnostic criteria. Sleep 2003;26:A338. [7] Vetrugno R, D’Angelo R, Montagna P. Periodic limb movements in sleep and periodic limb movement disorder. Neurol Sci 2007;28(Suppl.1):S9–S14. [8] Trenkwalder C. Restless Legs Syndrome: overdiagnosed or underdiagnosed? Nat Clin Pract 2007;3:474–5. [9] Allen RP, Walters AS, Montplaisir J, Hening W, Myers A, Bell TJ, et al. Restless legs syndrome prevalence and impact: REST general population study. Arch Int Med 2005;165(11):1286–92. [10] Hening W, Walters AS, Allen RP, Montplaisir J, Myers A, FeriniStrambi L. Impact, diagnosis and treatment of restless legs syndrome (RLS) in a primary care population: the REST (RLS epidemiology, symptoms, and treatment) primary care study. Sleep Med 2004;5(3):237–46. [11] Hening WA, Allen RP, Washburn M, Lesage S, Earley CJ. Validation of the Hopkins telephone diagnostic interview for restless legs syndrome. Sleep Med 2008;9:283–9. [12] Hening WA, Allen RP, Thanner S, Washburn T, Heckler D, Walters AS, et al. The Johns Hopkins telephone diagnostic interview for the restless legs syndrome: preliminary investigation for validation in a multi-center patient and control population. Sleep Med 2003;4:137–41. [13] Benes H, Walters AS, Allen RP, Hening WA, Kohnen R. Definition of the restless legs syndrome, how to diagnose it and how to differentiate it from RLS mimics. Mov Disord 2007;22(18):S485–94. [14] The International Restless Legs Syndrome Study Group. Validation of the International Restless Legs Syndrome Study Group rating scale for restless legs syndrome. Sleep Med 2003;4(2):121– 132. [15] Kohnen R, Stiasny-Kolster K, Oertel WH, Benes H, Trenkwalder C. Severity rating of restless legs syndrome: validation of the RLS-6 scales. Sleep 2004;27(Abstract Suppl.):A342. [16] Allen RP, Earley CJ. Validation of the Johns Hopkins restless legs severity scale. Sleep Med 2001;2(3):239–42. [17] WHO Info Package. Mastering depression in primary care. Frederiksborg: World Health Organization, Regional Office for Europe, Psychiatric Research Unit; 1998. [18] Goertelmeyer R. On the development of a standardized sleep inventory for the assessment of sleep. In: Kubicki S, Herrmann WM, editors. Methods of sleep research. Gustav Fischer: Stuttgart; 1985. [19] Bullinger M, Kirchberger I. SF-36 Health Questionnaire [German]. Goettingen: Hogrefe; 1998.

H. Benesˇ, R. Kohnen / Sleep Medicine 10 (2009) 515–523 [20] Benes H, Kaulfuss L, Muchalski T, Kohnen R. Disturbed sleep. Are there differences in subjective symptom assessments between RLS patients and patients with primary insomnia? Sleep Med 2005;6(Suppl. 2):S119. [21] Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 1982;143(1):29–36. [22] Campbell DT, Fiske JL. Convergent and discriminant validation by the multitrait multimethod matrix. Psychol Bull 1959;56:85–105. [23] Cronbach LJP. Coefficient alpha and the internal structure of tests. Psychometrika 1951;16:297–334. [24] Benes H, Von Eye A, Kohnen R. Empirical evaluation of the accuracy of diagnostic criteria for the Restless legs syndrome. Sleep Med 2008. doi:10.1016/j.sleep.2008.06.007. [25] Hening WA, Allen RP, Chaudhuri KR, Hornyak M, Lee HB, Winkelman J, et al. Clinical significance of RLS. Mov Disord 2007;22(18):S395–400. Ref type: Abstract.

523

[26] Hening WA. Subjective and objective criteria in the diagnosis of the restless legs syndrome. Sleep Med 2004;5(3):285–92. [27] Stiasny-Kolster K, Kohnen R, Carsten MJ, Trenkwalder C, Oertel WH. Validation of the ‘‘L-DOPA test” for diagnosis of restless legs syndrome. Mov Disord 2006;21:1333–9. [28] Walters AS. Restless legs syndrome and periodic limb movements in sleep. Continuum Lifelong Learning Neurol 2007;13: 115–38. [29] Ferri R, Lanuzza B, Cosentino FI, Iero I, Tripodi M, Spada RS, et al. A single question for the rapid screening of restless legs syndrome in the neurological clinical practice1. Eur J Neurol 2007;14(9):1016–21. [30] Wittchen HU. Reliability and validity studies of the WHO– Composite International Diagnostic Interview (CIDI): a critical review. J Psychiatr Res 1994;28(1):57–84.