- Email: [email protected]
On relative frequency estimation of transcranial magnetic stimulation motor threshold
Clinical Neurophysiology 123 (2012) 2319–2320
Contents lists available at SciVerse ScienceDirect
Clinical Neurophysiology journal homepage: www.elsevier.com/locate/clinph
Letter to the Editor On relative frequency estimation of transcranial magnetic stimulation motor threshold
In recent IFCN guidelines for diagnostic transcranial magnetic stimulation (Groppa et al., 2012) relative frequency estimation of cortical motor threshold (CMT) is promoted (e.g. the lowest stimulus intensity which produces more than 5 MEPs with 10 stimuli). Although such an estimation procedure is widely used within TMS laboratories (Groppa et al., 2012) the mathematical properties of the relative frequency estimator for CMT have never been rigorously evaluated so far. I performed such an analysis on ‘‘mathematical subjects’’ that are represented by a sigmoidal increase (cumulative Gaussian) of motor evoked potential (MEP) occurrence probability with increasing stimulus strength. The true CMT is defined as the unique stimulus strength for which the probability of MEP occurrence is 0.5. The stimulus strength increase that is necessary to increase MEP probability from 0.5 to 0.84 (the ‘‘threshold spread’’) was chosen to be 7% of the respective true CMT. These properties ensure that the mathematical subjects behave almost identically to real subjects in the concrete laboratory environment (Awiszus, 2003). In total, 7501 mathematical subjects were analyzed (true CMTs ranging from 25% maximum stimulator output (MSO) to 100% MSO in steps of 0.01% MSO) covering the entire range of CMTs that may be encountered in a clinical situation. Application of a particular estimation procedure on a mathematical or real subject yields a concrete CMT estimate. I will call such a CMT estimate diagnostically acceptable, if it is closer to true CMT than to either a clearly subthreshold stimulus strength of 90% true threshold or to a clearly suprathreshold stimulus strength of 110% true threshold. Repeated application of the same estimation procedure on the same subject an infinite number of times (only possible on mathematical subjects with complete enumeration (Awiszus, 2011)) will characterize the distribution the estimator has for this subject and will allow to calculate exactly the probability to obtain a diagnostically acceptable estimator. These probability calculations were performed by constructing a tree of all possible outcomes for each investigated threshold estimation algorithm. For each stimulus requested by the algorithm two new branches are added to the tree, one corresponding to the case of observing an appropriate MEP for this stimulus and the other corresponding to the case of no adequate response. Probabilities for the new branches were assigned from the assumed response probabilities for a given mathematical subject. I will call a CMT estimation procedure sufficiently accurate for a given subject, if the probability to obtain a diagnostically acceptable estimator for this subject exceeds 0.95. A CMT estimation procedure will be called mathematically valid, if it is sufficiently accurate for all subjects with true CMTs in the range from 25% MSO to 100% MSO.
The first concrete CMT estimation procedure mentioned by Groppa et al. (2012) is ‘‘the smallest stimulus strength with (exactly) 5 MEPs out of 10 trials’’. This strict relative frequency criterion has nasty mathematical properties and there is a substantial probability that such a stimulus strength may not exist at all (see Fig. 3a in Awiszus, 2003). The analysis showed that there is no subject for which this estimator is sufficiently accurate and thus it is mathematically invalid. The IFCN guidelines (Groppa et al., 2012) promote a slightly modified relative frequency estimation procedure: the smallest stimulus strength with at least 5 MEPs out of 10 trials. The analysis of this procedure is summarized in Fig. 1 which gives the probability to obtain a diagnostically acceptable estimate as a function of the underlying true CMTs. It can be seen that this probability fluctuates wildly and there are mathematical subjects for which this procedure is sufficiently accurate (i.e. the probability exceeds 0.95). Unfortunately, only 47.6% of the mathematical subjects analyzed satisfy this criterion and therefore this relative frequency estimator is mathematically invalid as well. Reducing the number of stimuli per tested stimulus strength will worsen the situation. Analyzing the ‘‘smallest stimulus strength with at least 3 MEPs out of 6 trials’’ (suggested by Groppa et al., 2012) yields that this procedure is sufficiently accurate for only 0.8% of the analyzed subjects. The results for the ‘‘smallest stimulus strength with at least 10 MEPs out of 20 trials’’ (mentioned in previous IFCN guidelines (Rossini et al., 1994)) showed that even such an enormous investment of stimuli does not yield a mathematically valid relative frequency estimator as the procedure is sufficiently accurate for only 96.2% of the analyzed subjects. Moreover, for acquisition of the described relative frequency estimators it is essential to begin with 1% MSO and test all physically realizable stimulus strengths in ascending order until the prespecified relative frequency criterion (e.g. at least 5 positive responses out of 10 trials) is met. Such an algorithm is clearly unfeasible clinically and the relative frequency algorithm presented by Groppa et al. (2012) has no chance to obtain the specified relative frequency estimator despite the fact that it will certainly need more than 50 stimuli on average. In sharp contrast to relative frequency estimation, maximumlikelihood threshold tracking (Awiszus, 2003) with 20 stimuli was found to yield a CMT estimator that is sufficiently accurate for all subjects analyzed and thus is mathematically valid. However, if invalid a–priori assumptions and obsolete stopping rules are added to my method (as done by Qi et al. (2011)) the probability to obtain diagnostically unacceptable threshold estimates is increased for all mathematical subjects beyond 0.05 and thus the Qi et al. (2011) procedure is mathematically invalid. Strictly speaking, the results presented are valid only for mathematical subjects. However, mathematical subjects may be considered as a best-case scenario for a TMS researcher (no coil movement, no inadvertent voluntary contractions etc.) and the
1388-2457/$36.00 Ó 2012 International Federation of Clinical Neurophysiology. Published by Elsevier Ireland Ltd. All rights reserved. http://dx.doi.org/10.1016/j.clinph.2012.04.014
2320
Letter to the Editor / Clinical Neurophysiology 123 (2012) 2319–2320
mathematical validity. Fortunately, Groppa et al. (2012) recommend to use maximum-likelihood threshold tracking which is corroborated by a rigorous mathematical proof (although only for (Awiszus, 2003)), while the contradictory recommendation of other TMS guidelines (Fitzgerald and Daskalakis, 2012) to insist on strict relative frequency estimation of CMT must be regarded as obsolete. References
Fig. 1. Summary of the analysis of the CMT estimator promoted by Groppa et al. (2012), the smallest stimulus strength with at least 5 MEPs out of 10 trials. The probability to obtain a diagnostically acceptable threshold estimate as a function of true CMT of the analyzed mathematical subjects is shown.
proportion of mathematical subjects with a sufficiently accurate threshold estimator should be regarded as a somewhat optimistic overestimate for the corresponding proportion in the clinical situation. Clearly, mathematical validity of a CMT estimation procedure is a necessary condition for clinical validity. Groppa et al. (2012) conclude that ‘‘the methods described above (where relative frequency estimation and the Qi et al. (2011) procedure are included) have a validated scientific background and can be used in a clinical setting because they provide a CMT estimation that is sufficiently accurate for diagnostic purposes’’. However, the results presented show that these conclusions are not justified as all variants of relative frequency CMT estimation procedures and the Qi et al. (2011) procedure lack
Awiszus F. TMS and threshold hunting. Suppl Clin Neurophysiol 2003;56:13–23. Awiszus F. Fast estimation of transcranial magnetic stimulation motor threshold: is it safe? Brain Stimul 2011;4:58–9. Fitzgerald PB, Daskalakis ZJ. A practical guide to the use of repetitive transcranial magnetic stimulation in the treatment of depression. Brain Stimul 2012. http:// dx.doi.org/10.1016/j.brs.2011.03.006. Groppa S, Olivero A, Eisen A, Quartarone A, Cohen LG, Mall V, et al. A practical guide to diagnostic transcranial magnetic stimulation: report of an IFCN committee. Clin Neurophysiol 2012;123:858–82. Qi F, Wu AD, Schweighofer N. Fast estimation of transcranial magnetic stimulation motor threshold. Brain Stimul 2011;4:50–7. Rossini PM, Barker AT, Berardelli A, Caramia MD, Caruso G, Cracco RQ, et al. Noninvasive electrical and magnetic stimulation of the brain, spinal cord and roots: basic principles and procedures for routine clinical application. Report of an IFCN committee. Electroencephalogr Clin Neurophysiol 1994;91:79–92.
Friedemann Awiszus Neuromuscular Research Group at the Department of Orthopaedics, Otto-von-Guericke University, Magdeburg, Germany. Tel.: +49 391 6714067 E-mail address: [email protected] Available online 15 May 2012
Contents lists available at SciVerse ScienceDirect
Clinical Neurophysiology journal homepage: www.elsevier.com/locate/clinph
Letter to the Editor On relative frequency estimation of transcranial magnetic stimulation motor threshold
In recent IFCN guidelines for diagnostic transcranial magnetic stimulation (Groppa et al., 2012) relative frequency estimation of cortical motor threshold (CMT) is promoted (e.g. the lowest stimulus intensity which produces more than 5 MEPs with 10 stimuli). Although such an estimation procedure is widely used within TMS laboratories (Groppa et al., 2012) the mathematical properties of the relative frequency estimator for CMT have never been rigorously evaluated so far. I performed such an analysis on ‘‘mathematical subjects’’ that are represented by a sigmoidal increase (cumulative Gaussian) of motor evoked potential (MEP) occurrence probability with increasing stimulus strength. The true CMT is defined as the unique stimulus strength for which the probability of MEP occurrence is 0.5. The stimulus strength increase that is necessary to increase MEP probability from 0.5 to 0.84 (the ‘‘threshold spread’’) was chosen to be 7% of the respective true CMT. These properties ensure that the mathematical subjects behave almost identically to real subjects in the concrete laboratory environment (Awiszus, 2003). In total, 7501 mathematical subjects were analyzed (true CMTs ranging from 25% maximum stimulator output (MSO) to 100% MSO in steps of 0.01% MSO) covering the entire range of CMTs that may be encountered in a clinical situation. Application of a particular estimation procedure on a mathematical or real subject yields a concrete CMT estimate. I will call such a CMT estimate diagnostically acceptable, if it is closer to true CMT than to either a clearly subthreshold stimulus strength of 90% true threshold or to a clearly suprathreshold stimulus strength of 110% true threshold. Repeated application of the same estimation procedure on the same subject an infinite number of times (only possible on mathematical subjects with complete enumeration (Awiszus, 2011)) will characterize the distribution the estimator has for this subject and will allow to calculate exactly the probability to obtain a diagnostically acceptable estimator. These probability calculations were performed by constructing a tree of all possible outcomes for each investigated threshold estimation algorithm. For each stimulus requested by the algorithm two new branches are added to the tree, one corresponding to the case of observing an appropriate MEP for this stimulus and the other corresponding to the case of no adequate response. Probabilities for the new branches were assigned from the assumed response probabilities for a given mathematical subject. I will call a CMT estimation procedure sufficiently accurate for a given subject, if the probability to obtain a diagnostically acceptable estimator for this subject exceeds 0.95. A CMT estimation procedure will be called mathematically valid, if it is sufficiently accurate for all subjects with true CMTs in the range from 25% MSO to 100% MSO.
The first concrete CMT estimation procedure mentioned by Groppa et al. (2012) is ‘‘the smallest stimulus strength with (exactly) 5 MEPs out of 10 trials’’. This strict relative frequency criterion has nasty mathematical properties and there is a substantial probability that such a stimulus strength may not exist at all (see Fig. 3a in Awiszus, 2003). The analysis showed that there is no subject for which this estimator is sufficiently accurate and thus it is mathematically invalid. The IFCN guidelines (Groppa et al., 2012) promote a slightly modified relative frequency estimation procedure: the smallest stimulus strength with at least 5 MEPs out of 10 trials. The analysis of this procedure is summarized in Fig. 1 which gives the probability to obtain a diagnostically acceptable estimate as a function of the underlying true CMTs. It can be seen that this probability fluctuates wildly and there are mathematical subjects for which this procedure is sufficiently accurate (i.e. the probability exceeds 0.95). Unfortunately, only 47.6% of the mathematical subjects analyzed satisfy this criterion and therefore this relative frequency estimator is mathematically invalid as well. Reducing the number of stimuli per tested stimulus strength will worsen the situation. Analyzing the ‘‘smallest stimulus strength with at least 3 MEPs out of 6 trials’’ (suggested by Groppa et al., 2012) yields that this procedure is sufficiently accurate for only 0.8% of the analyzed subjects. The results for the ‘‘smallest stimulus strength with at least 10 MEPs out of 20 trials’’ (mentioned in previous IFCN guidelines (Rossini et al., 1994)) showed that even such an enormous investment of stimuli does not yield a mathematically valid relative frequency estimator as the procedure is sufficiently accurate for only 96.2% of the analyzed subjects. Moreover, for acquisition of the described relative frequency estimators it is essential to begin with 1% MSO and test all physically realizable stimulus strengths in ascending order until the prespecified relative frequency criterion (e.g. at least 5 positive responses out of 10 trials) is met. Such an algorithm is clearly unfeasible clinically and the relative frequency algorithm presented by Groppa et al. (2012) has no chance to obtain the specified relative frequency estimator despite the fact that it will certainly need more than 50 stimuli on average. In sharp contrast to relative frequency estimation, maximumlikelihood threshold tracking (Awiszus, 2003) with 20 stimuli was found to yield a CMT estimator that is sufficiently accurate for all subjects analyzed and thus is mathematically valid. However, if invalid a–priori assumptions and obsolete stopping rules are added to my method (as done by Qi et al. (2011)) the probability to obtain diagnostically unacceptable threshold estimates is increased for all mathematical subjects beyond 0.05 and thus the Qi et al. (2011) procedure is mathematically invalid. Strictly speaking, the results presented are valid only for mathematical subjects. However, mathematical subjects may be considered as a best-case scenario for a TMS researcher (no coil movement, no inadvertent voluntary contractions etc.) and the
1388-2457/$36.00 Ó 2012 International Federation of Clinical Neurophysiology. Published by Elsevier Ireland Ltd. All rights reserved. http://dx.doi.org/10.1016/j.clinph.2012.04.014
2320
Letter to the Editor / Clinical Neurophysiology 123 (2012) 2319–2320
mathematical validity. Fortunately, Groppa et al. (2012) recommend to use maximum-likelihood threshold tracking which is corroborated by a rigorous mathematical proof (although only for (Awiszus, 2003)), while the contradictory recommendation of other TMS guidelines (Fitzgerald and Daskalakis, 2012) to insist on strict relative frequency estimation of CMT must be regarded as obsolete. References
Fig. 1. Summary of the analysis of the CMT estimator promoted by Groppa et al. (2012), the smallest stimulus strength with at least 5 MEPs out of 10 trials. The probability to obtain a diagnostically acceptable threshold estimate as a function of true CMT of the analyzed mathematical subjects is shown.
proportion of mathematical subjects with a sufficiently accurate threshold estimator should be regarded as a somewhat optimistic overestimate for the corresponding proportion in the clinical situation. Clearly, mathematical validity of a CMT estimation procedure is a necessary condition for clinical validity. Groppa et al. (2012) conclude that ‘‘the methods described above (where relative frequency estimation and the Qi et al. (2011) procedure are included) have a validated scientific background and can be used in a clinical setting because they provide a CMT estimation that is sufficiently accurate for diagnostic purposes’’. However, the results presented show that these conclusions are not justified as all variants of relative frequency CMT estimation procedures and the Qi et al. (2011) procedure lack
Awiszus F. TMS and threshold hunting. Suppl Clin Neurophysiol 2003;56:13–23. Awiszus F. Fast estimation of transcranial magnetic stimulation motor threshold: is it safe? Brain Stimul 2011;4:58–9. Fitzgerald PB, Daskalakis ZJ. A practical guide to the use of repetitive transcranial magnetic stimulation in the treatment of depression. Brain Stimul 2012. http:// dx.doi.org/10.1016/j.brs.2011.03.006. Groppa S, Olivero A, Eisen A, Quartarone A, Cohen LG, Mall V, et al. A practical guide to diagnostic transcranial magnetic stimulation: report of an IFCN committee. Clin Neurophysiol 2012;123:858–82. Qi F, Wu AD, Schweighofer N. Fast estimation of transcranial magnetic stimulation motor threshold. Brain Stimul 2011;4:50–7. Rossini PM, Barker AT, Berardelli A, Caramia MD, Caruso G, Cracco RQ, et al. Noninvasive electrical and magnetic stimulation of the brain, spinal cord and roots: basic principles and procedures for routine clinical application. Report of an IFCN committee. Electroencephalogr Clin Neurophysiol 1994;91:79–92.
Friedemann Awiszus Neuromuscular Research Group at the Department of Orthopaedics, Otto-von-Guericke University, Magdeburg, Germany. Tel.: +49 391 6714067 E-mail address: [email protected] Available online 15 May 2012