Response to Article by American Congress of Rehabilitation Medicine, Brain-Injury-Interdisciplinary Special Interest Group, Disorders of Consciousness Task Force

1181

DEPARTMENTS

Letters to the Editor Disorders of Consciousness: Recommendations Are Welcome but Further Guidance is Needed We welcome the recommendations for assessment scales for disorders of consciousness (DOC) by Seel et al,1 which represent progress toward ensuring high-quality assessment for these patients. However, insufficient emphasis on the importance of repeated assessments2 is a limitation. Together with the emphasis on “reasonable time” for administration, there is a risk that behaviors inconsistent with vegetative state may continue to be missed. The Recommendations do state that “extended or repeated assessment with a DOC scale is likely to improve diagnostic accuracy.” However, no concrete guidance is given regarding the recommended number of such assessments in clinical practice. “Extended or repeated” may be variously interpreted as anything from 2 assessments on the same day, upward to daily assessments over a number of weeks. A helpful addition to the Recommendations would be a statement of current expert opinion regarding a minimum number of assessments and/or time spent in behavioral assessment before a conclusion can be reached about the patient’s functional level. Most assessment scales (Sensory Modality And Rehabilitation Technique [SMART]3 excepted) do not, themselves, make a specific recommendation on this point. Indeed, this aspect of SMART is overlooked in the Recommendations, which state somewhat misleadingly that assessment time for SMART is 60 minutes or more. In fact, 10 assessments, each of 60 minutes, are required. A clearer indication of the role of assessment scales in the broader clinical diagnostic process would also be welcome. Even the most recent of previous guidelines4 on DOC diagnosis are now several years old and predate developments in magnetic resonance imaging and neurophysiologic techniques. The important point in the Recommendations, that current evidence is insufficient to allow a recommendation on the use of these scales for diagnostic classification, is absent from the conclusions in the abstract.1 It is especially important to highlight this when, for example, the Coma Recovery Scale-Revised (CRS-R) specifically yields a diagnostic classification. Thus, the CRS-R is recommended for patient assessment, but not for diagnosing a patient with a DOC. This somewhat subtle distinction is at risk of being overlooked by clinicians with less experience in assessment of DOC, presumably a group to whom the Recommendations are directed. We join others5 in a call for a broader update of guidance on diagnosis of DOC, including but not limited to, behavioral assessment. Alison K. Godbolt, MRCP, MD Staffan Stenson, Clin Psych Maria Winberg, OT reg Gunilla E. Frykberg, PhD Christer Tengvar, PhD Department of Rehabilitation Medicine University Hospital Uppsala, and Institute of Neuroscience Uppsala University Uppsala, Sweden

Disclosure: No commercial party having a direct financial interest in the results of the research supporting this letter has or will confer a benefit on the authors or on any organization with which the authors are associated. References 1. American Congress of Rehabilitation Medicine, Brain InjuryInterdisciplinary Special Interest Group, Disorders of Consciousness Task Force, Seel RT, Sherer M, et al. Assessment scales for disorders of consciousness: evidence-based recommendations for clinical practice and research. Arch Phys Med Rehabil 2010;91: 1795-813. 2. Gill-Thwaites H. Lotteries, loopholes and luck: misdiagnosis in the vegetative state patient. Brain Inj 2006;20:1321-8. 3. Gill-Thwaites H, Munday R. The Sensory Modality Assessment and Rehabilitation Technique (SMART): a valid and reliable assessment for vegetative state and minimally conscious state patients. Brain Inj 2004;18:1255-69. 4. Royal College of Physicians. The vegetative state: guidance on diagnosis and management. Available at: http://bookshop.rcplondon. ac.uk/contents/47a262a7-350a-490a-b88d-6f58bbf076a3.pdf. Accessed February 20, 2011. 5. Coleman MR, Bekinschtein T, Monti MM, Owen AM, Pickard JD. A multimodal approach to the assessment of patients with disorders of consciousness. Prog Brain Res 2009;177:231-48.

doi:10.1016/j.apmr.2011.03.034

Response to Article by American Congress of Rehabilitation Medicine, Brain-Injury-Interdisciplinary Special Interest Group, Disorders of Consciousness Task Force As an expert clinician having completed in excess of 230 Sensory Modality Assessment and Rehabilitation Technique (SMART) assessments during 23 years working in this specialist field,1 I believe the article by Seel et al2 provides a thorough overview of the validity features of the tools. However, there are some significant shortcomings which must be highlighted. While the authors maintain that no party had any direct interest in the results of the research, it is clear that the inclusion of the authors of the Coma Recovery Scale3 and the Disorders of Consciousness Scale4 on the review special interest group (SIG) must certainly have biased the results. Any comparison of such tools must remain totally independent from the authors of the tools under review, because at the very least, they had the advantage of being well placed to clarify any issues raised concerning the details of their assessments. I am concerned that the SIG felt that they were able to complete a thorough and balanced review of the complex and robust SMART tool, when the SIG didn’t insist on having access to the full copy of the SMART manual and techniques, nor a complete set of the paperwork. Additionally, the authors made no contact to ask any questions or to establish the content of our intensive, mandatory 5-day training course and extensive computerized data analysis system. As a result, the SMART was misrepresented and Arch Phys Med Rehabil Vol 92, July 2011

1182

LETTERS TO THE EDITOR

the details of the tools’ inter- and intrarater reliability and diagnostic outcome features were incorrect. The article also omits to identify the other key factors, which are of equal value to the validity of the tools. These include: the need for the assessor to have the knowledge and experience in this specific field; the need for consistency of the assessor to gain familiarity with the patients’ behavioral repertoire; standardization of the assessment process and the stimuli used; clear guidelines as to the frequency of assessments applied; the need for mandatory involvement of the family and team; and finally the impact of variations in the patients’ positioning, influencing environmental factors, time of day, and medical and nutritional status during each assessment. These are all mandatory requirements for SMART but are not fully inclusive in other assessment tools. When Nelson and Bernat5 asked a group of neurologists to comment on the importance of 10 features of a vegetative state (VS) to provide an operational definition, they found marked confusion and lack of consensus. Without the benefit of training to competently discriminate behaviors and responses observed in VS and minimally conscious state patients, an assessor cannot reliably reach the correct diagnosis based on a sporadic assessment process. Conversely, SMART would be administered by a trained assessor, meeting the person specification, and assessing the patient on 10 occasions, before reaching a diagnosis. Recommending a single tool above all others, based on the narrow parameters presented in this article, is misleading. Professionals must have the insight to identify that there are no short cuts in the assessment of this unique patient group. Failure to apply the most comprehensive assessment tools will be doing the patient an injustice. Helen Gill-Thwaites, MBE, MSc Royal Hospital for Neuro Disability London, United Kingdom Disclosure: No commercial party having a direct financial interest in the results of the research supporting this letter has or will confer a benefit on the authors or on any organization with which the authors are associated. References 1. Gill-Thwaites H. The Sensory Modality Assessment and Rehabilitation Technique–a tool for the assessment and treatment of patients with severe brain injury in a vegetative state. Brain Inj 1997;11:723-34. 2. American Congress of Rehabilitation Medicine, Brain InjuryInterdisciplinary Special Interest Group, Disorders of Consciousness Task Force, Seel RT, Sherer M, et al. Assessment scales for disorders of consciousness; evidence-based recommendations for clinical practice and research. Arch Phys Med Rehabil 2010;91: 1795-813. 3. Giacino JT, Kalmer K, Whyte J. The JKF Coma Recovery ScaleRevised; measurement characteristics and diagnostic utility. Arch Phys Med Rehabil 2004;85:2020-9. 4. Pape TL, Heinemann AW, Kelly JP, Hurder AG, Lundgren S. A measure of neurobehavioural functioning after coma. Part 1: theory, reliability, and validity of Disorders of Consciousness Scale. J Rehabil Res Dev 2005;42:1-17. 5. Nelson WA, Bernat JL. Decisions to withhold or terminate treatment. Neurol Clin 1989;7:759-74.

doi:10.1016/j.apmr.2011.03.033

Arch Phys Med Rehabil Vol 92, July 2011

The Authors Respond After publication of our systematic review on the disorders of consciousness (DOC) assessment scales,1 Godbolt et al and Gill-Thwaites expressed several concerns in letters to the editor. The concerns fall into 2 categories—making additional evaluations and recommendations, and the review process itself—which we address below. Godbolt was concerned that “insufficient emphasis” was placed on the importance of repeated assessment of persons with DOC. While Godbolt seems to agree with our statement in the discussion section that “extended or repeated assessment with a DOC scale is likely to improve diagnostic accuracy,” she felt that we should have included “a statement of current expert opinion regarding a minimum number of assessments and/or time spent in behavioral assessment.” We agree that recommendations on the number, duration, and time period of assessments for establishing functional level in persons with DOC would be valuable, but our systemic review process requires that formal recommendations be based on empirical data. We did not find any studies that allowed us to make specific recommendations regarding the number and duration of assessments that would optimize the sensitivity and specificity of DOC scales. Gill-Thwaites was concerned that assessor knowledge and experience; using the same assessor; standardization of the assessment process; mandatory involvement of the family and team; and variations in patient positioning, environmental factors, timing of assessment, and medical status were not considered when evaluating the scales. Our task force evaluated the standardization of scale administration and scoring procedures, which was a critical criterion in determining whether scales were recommended as an assessment option. While examiner characteristics might impact scale use and controlling for potential confounds is clinically prudent, these factors are fairly independent from the scales themselves. Further, the length, breadth, or complexity of any scale, in and of itself, does not ensure its’ standardization, validity, reliability, practicality, or other qualities. Ultimately, the importance of assessor knowledge and experience, the impact of potential confounding factors on assessment, and the ideal type and amount of training on scale use are all empirical questions that were not scientifically addressed in any studies we reviewed. We agree that empirical answers to such research questions would be valuable. With regard to the review process, Gill-Thwaites expressed concerns that our task force did not acknowledge or manage conflicts of interest. Gill-Thwaites alleges that our task force maintains that “no party had direct interest in the results of the research.” She further alleges that because a few task force members were also authors of reviewed DOC assessment scales, their inclusion on the task force “must certainly have biased the results.” Our review followed the published clinical guideline development process of the American Academy of Neurology that has been used to conduct dozens of published reviews and guidelines.2 The standard process in which we acknowledged and addressed task force members’ potential conflicts was documented on the first page of the article. We identified 3 task force members who were authors of 2 of the DOC scales reviewed and 1 member who was a coauthor on 1 of the studies reviewed. Because both scales are in the public domain with no plans for commercial distribution, we reported that “no party . . . has or will confer a financial benefit.” We then detailed the steps our 14 member task force took to manage potential conflicts including balancing rater pairs with at least