- Email: [email protected]
NoGo task in ADHD subjects
Accepted Manuscript Title: Test-retest reliability of ERP components: A short-term replication of a visual Go/NoGo task in ADHD subjects Author: Kyveli Kompatsiari Gian Candrian Andreas Mueller PII: DOI: Reference:
S0304-3940(16)30080-5 http://dx.doi.org/doi:10.1016/j.neulet.2016.02.012 NSL 31840
To appear in:
Neuroscience Letters
Received date: Revised date: Accepted date:
10-11-2015 2-2-2016 4-2-2016
Please cite this article as: Kyveli Kompatsiari, Gian Candrian, Andreas Mueller, Test-retest reliability of ERP components: A short-term replication of a visual Go/NoGo task in ADHD subjects, Neuroscience Letters http://dx.doi.org/10.1016/j.neulet.2016.02.012 This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Test-retest reliability of ERP components: A short-term replication of a visual Go/NoGo task in ADHD subjects
Kyveli Kompatsiaria, Gian Candriana, Andreas Muellera1
a
Brain and Trauma Foundation Grisons, Chur, Switzerland
1
Corresponding Author: Andreas Mueller, Brain and Trauma Foundation Grisons, Poststrasse 22, CH-7000 Chur, [email protected]
Highlights
This paper presents the results of a short-term test-retest reliability study of ERPs and Independent Components (ICs) in children with ADHD during a visual Go/NoGo paradigm.
Importantly, ICs which are directly linked to distinct cognitive processes, show comparable results with the raw ERPs.
In most cases, amplitudes show ‘good’ to ‘excellent’ test–retest reliability results, while latencies varied from a ‘poor’ to an ‘excellent’ level.
The obtained test-retest reliability results of the ADHD subjects are similar with previous studies on neurotypical samples.
Abstract Event-related potentials (ERPs) have been widely used to investigate brain functioning in children with Attention Deficit Hyperactivity Disorder (ADHD) in both research and diagnostic settings. To ensure the efficiency of ERP techniques in ADHD diagnosis and in longitudinal observational studies, the test-retest reliability of the affected population must be validated. Thus, the present article assesses the short-term testretest reliability of certain early and late ERPs (i.e. P1, N1, N2, P2, P3), as well as independent components (ICs) decomposed from the above mentioned ERPs (IC P3 Go, IC P3 NoGo early, IC P3 NoGo late) relevant to ADHD, through the Intraclass Correlation Coefficient (ICC). More specifically, we employ a cued visual Go/NoGo paradigm for recording ERPs from 22 children with ADHD (mean age 12.2), twice within 30 minutes. Amplitudes and latencies are calculated by the 'peak amplitude' method and by a variation of the fractional area. Results for amplitudes lie mostly within the 'good' and 'excellent' range for both measurement methods, while ICC for latencies is more variable ranging from 'poor' to ‘excellent’ results. Crucially, the ICs, which are associated with distinct functionally independent processes of the executive attentional
system have shown a comparable test-retest reliability with the raw ERPs. Our results are consistent with other reliability studies of neurotypical population in the literature, and as such, consist initial evidence that ERPs could be reliable neurophysiological markers for the ADHD population.
Keywords Test-retest reliability; Event-related potentials; Independent Components, ADHD; Visual Continuous Performance Task, Intraclass Correlation Coefficient
1. Introduction A variety of tasks have been used to elicit event-related potentials (ERPs) related to human sensory, motor, and cognitive functions. The non-invasiveness of the technique and the possibility of a direct measurement of neurocognitive functions with an excellent temporal resolution have made ERPs one of the most widely used methods in cognitive neuroscience research. Studies exploring various aspects of brain functioning associated with psychiatric disorders report ERP differences related to sensory and cognitive-processing [1]. ADHD is a prevalent developmental disorder, characterized by varying levels of inattention, hyperactivity, and impulsivity symptoms, where deviances have been reported for several sensory and cognitive ERP components [2]. Using the Go/NoGo paradigm, Smith et al. found clear differences in the early processing components (P1, N1, P2) of responses to Go and NoGo stimuli, which they interpreted as reflecting problems with sensory registration and identification of stimuli for children with ADHD [3]. Several studies have reported N2 (NoGo) and P3 waves (Go/NoGO) to be deviant in ADHD [4-9]. N2 and P3 NoGo waves have been associated with inhibition processes. Due to its late appearance, P3 has been further related to
evaluation of inhibitory processes and a voluntary decision to withhold the motor response [10-17]. The parietal P3 Go has been suggested to be similar to P3b elicited in oddball paradigms. P3b amplitude is said to reflect the allocation of attentional resources, and latency reflects classification speed. P3b is also argued to reflect a link between perceptual classification and response selection, i.e., monitoring whether the stimulus is classified correctly to the appropriate response [18-19]. Independent components (ICs) have also been studied in ADHD. In a study on the classification of ADHD and neurotypical individuals, Mueller et al., reported differences in the sensory components, as well as deviances in latencies and amplitudes of ICs derive from P3 Go component with a parietal distribution (IC P3 Go) and from P3 NoGo component (IC P3 NoGo early, IC P3 NoGo late) with a central and frontocentral distribution, respectively [9]. The IC P3 NoGo late has been associated with action monitoring [2021]. Recent research findings indicate a strong correlation between IC P3 NoGo early with energization, namely the ability to voluntarily invest attentional effort to optimize behavior for achieving a goal [21]. Additionally, recent studies show that IC P3 Go posterior, like P3b [18], reflects the process of concluding that a stimulus is classified correctly to the appropriate response, with the amplitude reflecting the degree of confidence in this conclusion [19]. Several studies have addressed the test-retest reliability of the ERP approach in a variety of cognitive tasks with the neurotypical population. The results range from ‘poor’ to ‘excellent’ within the session, over hours, days and years [17, 22-32]. The most frequently studied components are the P3b wave, error-related negativity (ERN), and Mismatch Negativity (MMN). In the majority of reliability studies, amplitudes generally show higher reliability than latencies2. Despite the fact that most of reliability research has insofar focused on young adults or on a specific age range, two studies have already provided with evidence that there are not systematic differences in test-retest reliabilities of ERP amplitudes between different age groups [30, 26]. Cassidy et al. mention ICC results ranging between 0.01 to 0.9 and covering a wide range of
2 For contradictory results see Brunner et al., Segalowitz and Barnes, Mei Hua Hall et al [17,22,32]
perceptual and cognitive components (P1, N1, N170, P3a, P3b, error-related negativity, error positivity, P400) [29]. Similar to the nature of our paradigm, Fallgatter et al. report ‘moderate’ to ‘excellent’ Intraclass Correlation Coefficient (ICC) results for P3 Go and P3 NoGo components using a CPT task twice [25]. Furthermore, a recent study addressing a 6-month replicate of a visual continuous Go/NoGo task (the same task used in the present study), reports excellent ICC for latencies and peaks of the P3 NoGo wave and two ICs decomposed from the P3 NoGo wave, the IC P3 NoGo early and the IC P3 NoGo late [17]. Studies generally use inconsistent stimuli, paradigms, test-retest measures, age groups, and between-session intervals, making it difficult to compare results. Importantly, to the best of our knowledge, there is a lack of test-retest reliability on the ADHD population. In this light, our study aims to serve as an initial step toward the validation of research and clinical efficiency of the ERP method on the ADHD population. Systematically investigating test-retest reliability in affected population could crucially inform both diagnostic and treatment practice. A short period of the test repetition was selected here for avoiding potential confounds due to exogenous factors, such as school and medication. Children with ADHD perform a visual continuous Go/NoGo task with a between-session interval of 30 minutes. Such a task targets crucial mechanisms in the affected population, such as sustained attention and inhibition. Regarding the specific components used to validate the reliability, all the above-mentioned components commonly used in ADHD research [2-19], are tested at a representative electrode with high activation of the component. More specifically, a number of earlier components are examined at the occipital electrode P1, N1 after the first stimulus of target relevant conditions (Go & NoGo conditions) and P2 at temporal electrode after second stimulus of NoGo condition. Additionally, the reliability of P3 Go component in the parietal electrode (Pz) after the second stimulus of Go condition and the independent component IC P3 Go with a parietal topography is calculated. Finally, we also test the N2 and P3 component at Cz electrode after the second stimulus of NoGo condition, and two ICs, namely IC P3 NoGo early and IC P3 NoGo late, decomposed from the raw ERP with central and fronto-central distributions, respectively.
2. Materials and methods 2. 1 Participants Our participants consist the first twenty five children participating in the 3-year study of Brain and Trauma Foundation, which focuses on the longitudinal investigation of biomarkers on ADHD subjects by measuring their brain activity over periods of six months. The test-retest recordings were conducted in August and September of 2015 with an interval of 30 minutes between sessions. All recordings were performed in morning hours, from 8:00AM - 14:00PM. The participants were referred to the study via advertisements to the medical doctors. They had been diagnosed with ADHD prior to their participation by an independent psychiatrist. DSM-IV criteria for ADHD were employed to confirm the diagnosis. Children were all diagnosed with either combined (6 children) or inattentive (19 children) subtypes of ADHD. Eight of the subjects were receiving medication, either Methylphenidate of 20mg or Concerta of 54mg, however, participants were required to refrain from taking medication during 24 hours before testing. Moreover, subjects suffering from a neurological disorder, severe comorbidities, or brain traumatic injury were not included in the study. The study was approved by the local ethics committee and written informed consent was obtained from the participants and their parents prior to their involvement. Subjects did not receive any compensation for their participation. 2.2 EEG recording The EEG was recorded using a NeuroAmp® x23 (BEE Medic GmbH), a PC-controlled 21-channel digital electroencephalographic system with a DC-coupling and 24-bit resolution. Electrodes were placed according to the International 10-20 system using an electrode cap with tin electrodes (Electro-cap International Inc.). Quantitative data was obtained using ERPrec software. The input signals, referenced to the linked earlobes, were filtered between 0.5 and 50 Hz and sampled at a rate of 250 Hz. Impedance was kept below 5 kOhm for all electrodes. Montage was changed to common average reference montage prior to data processing.
Blinking artefacts were corrected by zeroing the activation curves of individual independent components, acquired by application of ICA on raw EEG signal, which correspond to eye blinks. In addition, epochs of the filtered electroencephalogram with excessive amplitude (>120 μV) and/or excessive (threshold = z-score of 6) 0 to 3 Hz and 20 to 50 Hz band frequency activities were automatically excluded from further analysis. EEG was manually inspected to verify and exclude additional artefacts. Finally, trials with omission and commission errors were excluded. 2.3 Behavioral Task The performed task was a Visual Continuous Performance Task (VCPT) with 400 trials and 4 different conditions of equal probability appearing in a pseudorandomized order, and a duration of 20 minutes. Every trial consists of the presentation of a pair of images (100ms) with 1000ms inter-stimulus interval. The Go condition consists of an animal followed by a second animal (A-A). In this condition, the participant should press with the right hand the mouse as fast as possible. The NoGo condition consists of an animal followed by a plant (A-P), where the subjects should withhold their response. The third condition consists of two plants (P-P) and the fourth of a plant followed by an image (showing a human profession) together with an aberrant sound (P-H). The participants performed a preparatory practice phase to get familiarized with the task, ranging from 20 to 50 trials depending on their level of comprehension. In the middle of the task 5 minutes of rest was ensured for every subject. This study focused only on the trials consisting of target relevant stimuli, namely A-A and A-P. Participants were included only if they had more than 30 trials in each condition. Furthermore, only participants with more than 30 omission errors (not responding in go condition) were excluded as an indicator of excessively poor task engagement. In total 22 right-handed participants between 10 to 15 years old (Mean Age = 12.2, SD = 1.4, 4 female) remained for further analysis. 2.4 Decomposition of Collection of ERPs into Independent Components
When Independent Component Analysis (ICA) is applied to a collection of ERPs, a minimum amount of 100 individuals is required for a robust decomposition [20]. Having a limited number of participants, the ICA Infomax algorithm [33] was applied to a collection of individual ERPs of 185 (88 female) healthy subjects between 10-15 years old from HBi normative database, where the same behavioral task was used. ICA was performed using the EEGLAB Matlab toolbox [34]. The ICA input data was the two-dimensional 19-scalp-locations x 185-ERP-time-series matrix. The IC P3 Go, IC P3 NoGo early and IC P3 NoGo late components were extracted by applying ICA to the 1000 ms time interval after the second stimulus of the Go and No/Go trials respectively. The computed spatial filters were then applied to the individual ERPs. Under this analysis, The P3 NoGo wave could be decomposed into two independent components (IC P3 NoGo early and IC P3 NoGo late), while the P3 Go wave was decomposed in one independent component, IC P3 Go. More information about the features of the abovementioned ICs, i.e., latencies, topographies and functional meaning can be found in Kropotov et al., 2011 [20]. ICs were back-projected to the electrode with the highest activation of the component. 2.5 Measurement Measurements of latency and amplitude of raw ERP waves and ICs were performed with the 'peak amplitude' method by an ERP expert. As an additional objective method, a variation of the fractional area approach (vFA) is used for comparison reasons. More specifically, a latency window is defined for each extremum under analysis on the grand average curve. The window starts where the amplitude reaches the 50% amplitude value, between the extremum in question and its closest neighboring peak, and ends at the same amplitude value at the opposite side. As a following step, in this predefined latency window, the extremum value is selected on every individual curve and the area under/above the curve in this equally sized time window centered at the individual extremum is calculated.
2.6 Statistics The ICC was used as a test-retest reliability measure. ICC is an efficient method to measure test-retest reliability since it considers both the inter-subject and between-subjects variation. A two-way mixed effect model was employed with the setting of absolute agreement, showing to which degree the actual values are similar. The two-way mixed model was chosen because the error variance in measuring and scoring the ERPs is considered minimal while the error variance of the ERPs observations is regarded as unknown [35]. The reliability classification rates of Portney and Watkins was applied; namely, ICC below .50 is characterized as poor, moderate from .50 to .75, good from .75 to .90 and excellent when higher than .90 [36]. The means of the behavioral parameters and the ERP amplitudes and latencies between the two recordings were compared using two-tailed Paired-Sample T-Tests.
3. Results The mean epochs of A-A condition in the first session were 72 (±17) and 65 (±17) in the second, without showing a significant difference (t (21) = 0.93, p=0.36). The mean epochs of A-P condition were 76 (±14) in the first session and 72 (±15) in the second (t (21) = 1.5, p= 0.14). Furthermore, the average reaction time (first session: 415.11±68.5 vs second session: 420.48±83.3) does not differ significantly between the two measurements (t (21) = -1.29, p= 0.29). Finally, the average omission errors (first session: 7.4±7.48 vs second session: 8.27±6.22), also do not differ significantly between the two sessions (t (21) = -0.82, p= 0.42). The following tables 1, 2 show the statistical results of the ERP components between the two sessions for amplitudes and latencies respectively. More specifically, the first two columns illustrate the mean and standard deviation of the ERP parameters in the two recordings (only the peak method is used as a representative). The third column depicts the difference in the means (Mean Diff) between the two
measurements to investigate any systematic differences. The subsequent columns show the test-retest reliability results assessed with ICC for all the ERP components. The fourth represents the ‘peak’ method while the fifth the variation of the fractional area method, ‘vFA’. With regard to amplitudes, ADHD showed ‘good’ to ‘excellent’ reliability results in the majority of the components (Table 1). In contrast, latencies showed a higher variability, ranging from an ‘excellent’ to a ‘poor’ range. With respect to the ICs, reliability for amplitudes was slightly dropped compared to raw ERPs, but still remained within the ‘good’ range. Reliability regarding the latencies of the ICs is obviously larger than the raw ones. Using the vFA method, the reliability of the amplitudes showed a relatively small difference in comparison with the ‘peak’ method, while latencies showed better reliability on average. Moreover, regarding the variation of the reliability over time, a decline is clearly observed between the early and the later components. The means in the majority of the components did not differ significantly across sessions. However, the mean amplitudes of the raw P3 Go and NoGo waves, and the IC P3 NoGo early were significantly reduced from session 1 to session 2. Additionally, the mean latency was significantly increased in IC P3 Go and decreased in P3 NoGo. The grand average of the visual component at occipital electrode O2 for the two sessions and the topography of P1 peak latency are depicted in Figure 1. The grand average of the P2 NoGo wave at temporal electrode T6 and the topography of P2 peak latency are shown in Figure 2. The grand average of the P3 Go wave at Pz electrode and the topography of P3 peak latency are illustrated in Figure 3a. The P3 Go wave was decomposed in one IC, i.e., the IC P3 Go. The peak latency of IC P3 Go is around 300 ms after the 2nd stimulus (Go condition) and parietally distributed. The IC was back-projected to the Pz electrode where the highest activation of the component was situated. The topography and the activation curve of IC P3 Go are presented in Figure 3b for both sessions. The grand average of the P3 NoGo wave at Cz electrode and the topography of P3 peak latency is shown in Figure 4a. The P3 NoGo wave was decomposed in two ICs, i.e., the IC P3 NoGo early and IC P3 NoGo late. The IC P3 NoGo early is characterized by a positive deflection peaking around 330 ms after the onset of stimulus 2 and is centrally distributed. The IC P3 NoGo late shows
a positive peak around 380 ms post-stimulus 2 and has a central distribution as well. Both ICs were backprojected to the Cz electrode. The topographies and the activation curves of IC P3 NoGo early and IC NoGo late are presented in Figure 4b and 4c respectively.
Discussion The current study presents, to the best of our knowledge, the first short-term test-retest reliability results of several ERP components evoked during a Go/NoGo VCPT task, and relevant ICs derived from raw ERPs of Go/NoGo conditions, as tested on an ADHD sample. Amplitudes and latencies were extracted by the ‘peak amplitude’ method and by a variation of the fractional area. In total, in the major part of the components, subjects showed ‘good’ to ‘excellent’ reliability results concerning the amplitudes, while the latencies varied within the range of ‘poor’ to ‘excellent’ values. Our findings are in line with the majority of test-retest reliability studies and reviews, where both amplitudes and latencies were assessed [22-26,28,29]. Although it is difficult to compare reliability studies due to the variations in the tasks, reliability measures, and the test-retest interval, our study can be compared to some extent with three studies of reliability, Fallgatter et al. (2002), Brunner et al. (2013), and Cassidy et al. (2012) [25, 17, 29]. The similarity with the first two studies concern the nature of the performed task, the cognitive components evoked by this task and the ICs. Fallgatter et al. using a CPT task twice, report ‘good’ to ‘excellent’ ICC results for the peak amplitudes of P3 Go, P3 NoGo and ICs derived from these two conditions. Additionally, latencies show lower reliability values ranging from ‘good’ in NoGo condition to a not significant level in Go condition. These findings are in line with our findings where amplitudes lied in an identical range, whereas values for latencies were lower, with P3 Go latency showing also not significant results. However, Fallgatter et al. conducted a long-term reliability study for adult controls, using a different CPT task, and the model of ICC is not explicitly specified. In another long-term reliability study concerning young adult controls, Brunner et al., using the same VCPT task, the same way of ICA extraction and the same ICC measure, they report
good reliability (ICC > 0.75) for the amplitudes and excellent reliability (ICC > 0.9) for the latencies of ICs derived from NoGo condition. In our study, the reliability of the corresponding amplitudes lie also in the ‘good’ level, while reliability of latencies lie within a lower range compared to Brunner et al. study. In agreement with their work, in our study the area method had a greater positive impact on the reliability of the latencies. Concerning the earlier perceptual components, which are less affected by the task, we can compare our results with those found by Cassidy et al., where they state from moderate (latencies) to good (amplitudes) results using ICC in a visual oddball paradigm in P1 and N1 components (1-month inter-session interval) [29].Our study show higher values using both ‘peak’ and area methods, ranging from 0.75 to 0.96 for the early components. Importantly, in our study ICs show ‘good’ test-retest reliability results. Latencies of ICs are higher in comparison with raw ERPs probably due to their more concrete time decomposition. It is crucial to ensure a high test-retest reliability of ICs because each of them is linked to specific cognitive processes and would be beneficial to utilize them in a psychiatry context to reveal subject-specific traits (see Introduction part). In a number of cases, such as N2 NoGo peak and latency, P3 Go latency and P3 NoGo latency, a moderate to poor ICC is observed. In general, a drop in the reliability results is noticeable between early sensory and later components. Typically, the late components show higher variability than early ones because they are controlled more by endogenous factors than by the physical features of exogenous stimulation. According to Huffmeijer et al., at least 30 trials are required for early components while at least 60 trials, for later, broadly distributed components such as the P3 [31]. Later components reflecting more complex processes seem to require a higher number of trials for a larger reliability, which consists an intriguing part for children with ADHD. A systematic decline in amplitudes of later components is observed from the first to the second session. Certain components are significantly reduced, namely the amplitudes of the P3 Go, linked with allocation of attentional resources and stimulus classification monitoring, the P3 NoGo, related to the voluntary
decision to withhold a motor response, and IC P3 NoGo early, correlated with energization. One could argue that these components are generally linked with task engagement and conscious attentional effort and consequently, this observation might point to a slightly altered psychological state of the participants, for instance due to fatigue or adaptation effects. Although regular breaks were used to ensure the ease of the participants it might seem that a more extended retest time interval could alleviate this effect. This is the first study to explore the test-retest reliability of an ADHD sample on sensory and later ERP components and ICs decomposed from the raw ERPs. As stated by Nunnally and Bernstein [37], test-retest reliability of 0.80 is adequate for research experimental studies and 0.90 should be minimum for clinical purposes. Subsequently, according to our results most of the components could be safely used for research purposes, but not necessarily for clinical assessments. However, by addressing some limitations of the current study the stability of ERP components could be enhanced. For instance, the effect of the number of the averaged trials on the reliability of components in subjects with ADHD remains an important issue to be studied in future studies. Furthermore, although we think it is crucial to test the reliability of the affected population with a short repetition time in order to avoid long term interventional effects, considering a longer time period between tests might prove beneficial. In the latter case, side-effects from potential fatigue would be expected to be alleviated in a greater extent. Moreover, it would be highly informative to compare reliability results of an extended ADHD sample to neurotypicals, as well as to consider additional age groups. Concluding, we feel that more extensive research in this direction would be of great importance for encouraging the applicability of ERPs on clinical and research settings both on a diagnostic and treatment level, as for instance with regard to testing competing treatment strategies.
Conclusions To the best of our knowledge, this is the first study to examine the test-retest reliability of raw ERPs and independent components (ICs) on ADHD children. In particular, amplitudes showed ‘good’ to ‘excellent’ reliability results, while latencies varied from ‘poor’ to ‘excellent’ values. More importantly, ICs, crucial in psychiatry due to their association with different, functionally independent processes of the executive attentional system have shown a comparable test-retest reliability with the raw ERPs. Additionally, components of the ADHD subjects showed similar reliability results with previous studies on neurotypical samples. This study can serve as a first step toward the validation of ERPs and ICs as reliable biomarkers in ADHD. Our positive results indicate that such metrics, after further assessment, could be potentially employed by researchers and clinicians as an assistive tool in future diagnostic and treatment practice.
Conflict of interest statement Authors state no conflict of interest.
Acknowledgments The study was funded by Brain and Trauma Foundation. Additionally, the authors thank dr. Bernhard Wandernoth and Michael Schweikert for technical support. Finally, we are grateful to Gian Ruschetti for assistance in data acquisition and Dimitrios Bolis for scientific support.
References [1] J.D. Kropotov, Quantitative EEG, Event-Related Potentials and Neurotherapy, first ed. Academic Press, San Diego, 2009. [2] R.J. Barry, S.J. Johnstone, A.R. Clarke, A review of electrophysiology in attention-deficit/hyperactivity disorder II: Event-related potentials, Clin Neurophysiol. 114 (2003) 184-198. [3] J.L. Smith, S.J. Johnstone, R.J. Barry, Inhibitory processing during the Go/NoGo task: an ERP analysis of children with attention-deficit/hyperactivity disorder, Clin Neurophysiol. 115 (2004) 1320-1331. [4] M. Spronk, L.M. Jonkman, C. Kemner, Response inhibition and attention processing in 5- to 7-year-old children with and without symptoms of ADHD: an ERP study, Clin Neurophysiol. 119 (2008) 2738–2752. [5] M. Doehnert, D. Brandeis, K. Imhof, R. Drechsler, H.C. Steinhausen, Mapping attention-deficit/hyperactivity disorder from childhood to adolescence: no neurophysiologic evidence for a developmental lag of attention but some for inhibition, Biol Psychiatry. 7 (2010) 608–616. [6] D. Brandeis, T. Banaschewski, L. Baving, et al., Multicenter P300 brain mapping of impaired attention to cues in hyperkinetic children, J Am Acad Child Adolesc Psychiatry 41 (2002) 990–998. [7] J.R. Wiersema, H. Roeyers, ERP correlates of effortful control in children with varying levels of ADHD symptoms, J Abnorm Child Psychol. 37 (2009) 327–336. [8] S.J Johnstone, R.J Barry, A.R Clarke, Ten years on: a follow-up review of ERP research in attentiondeficit/hyperactivity disorder, Clin Neurophysiol. 124 (2013) 644–657. [9] A. Mueller, G. Candrian, J.D Kropotov, V.A Ponomarev, G.M. Baschera, Classification of ADHD patients on the basis of independent ERP components using a machine learning system, Nonlinear Biomed Phys. 4 (2010) S1. [10] A.J Fallgatter, W.K Strik, The NoGo-anteriorization as a neurophysiological standard-index for cognitive response control, Int J Psychophysiol. 32 (1999) 233-238. [11] A. Kok, J.R. Ramautar, M.B. De Ruiter, G.P.H. Band, K.R. Ridderinkhof, ERP components associated with successful and unsuccessful stopping in a stop-signal task, Psychophysiol. 41 (2004) 9–20.
[12] K.J. Bruin, A.A. Wijers, A.S. van Staveren, Response priming in a go/nogo task:do we have to explain the go/nogo N2 effect in terms of response activation instead of inhibition?, Clin Neurophysiol. 112 (2001) 1660–1671. [13] M. Liotti, S.R. Pliszka, R. Perez, D. Kothmann,, M.G Woldorff, Abnormal brain activity related to performance monitoring and error detection in children with children with ADHD, Cortex 41 (2005) 377–388. [14] M. Schmajuk, M. Liotti, L. Busse, M.G. Woldorff, Electrophysiological activity underlying inhibitory control processes in normal adults. Neuropsychologia 44 (2006) 384–395. [15] S. van Gaal, V.A.F. Lamme, J.J Fahrenfort, K.R. Ridderinkhof, Dissociable brain mechanisms underlying the conscious and unconscious control of behavior, J Cogn Neurosci. 23 (2011) 91–105. [16] P.D. Gajewski, M. Falkenstein, Effects of task complexity on ERP components in Go/Nogo tasks, Int J Psychophysiol. 87 (2013) 273–278. [17] J. F. Brunner, T.I. Hansen, O. Alexander, S. Toril, H. Asta, J. Kropotov, Long-term test-retest reliability of the P3 NoGo wave and two independent components decomposed from the P3 NoGo wave in a visual Go/NoGo task, Int J Psychophysiol. 89 (2013) 106–114. [18] R. Verleger, P. Jaskowski, & E. Wascher, Evidence for an integrative role of P3b in linking reaction to perception. J Psychophysiol. 19 (2005) 165–181. [19] I.E. Aasen, J.F. Brunner, Modulation of ERP components by task instructions in a cued 2 go/no-go task, Psychophysiol. (2015), in press. [20] J. D. Kropotov, V.A. Ponomarev, S. Hollup, A. Mueller, A., Dissociating action inhibition, conflict monitoring and sensory mismatch into independent components of event related potentials in GO/NOGO task. NeuroImage 57 (2011) 565–575. [21] J.F. Brunner, A. Olsen, I.E. Aasen, G.C. Løhaugen, A.K. Håberg, J. Kropotov Neuropsychological parameters indexing executive processes are associated with independent components of ERPs, Neuropsychologia 66 (2015) 144– 156. [22]S.J. Segalowitz, K.L. Barnes, The reliability of ERP components in the auditory oddball paradigm. Psychophysiol. 30 (1993) 451–459. [23] M. Tervaniemi, A. Lehtokoski, J. Sinkkonen, J. Virtanen, R.J. Ilmoniemi, R. Näätänen, Test-retest reliability of mismatch negativity for duration, frequency and intensity changes, Clin. Neurophysiol. 110 (1999) 1388-1393.
[24] C.A. Sandman, J.V. Patterson, The auditory event-related potential is a stable and reliable measure in elderly subjects over a 3 year period, Clin Neurophysiol. 111 (2000) 1427–1437. [25] A.J. Fallgatter, D.R. Aranda, A.J. Bartsch, M. J. Herrmann, Long-term Reliability of Electrophysiologic Response Control Parameters, Clin Neurophysiol. 19 (2002) 61–66. [26] K.B. Walhovd, A.M Fjell, One-year test–retest reliability of auditory ERPs in young and old adults. Int J Psychophysiol. 46 (2002) 29–40. [27] S.J. Segalowitz, D.L. Santesso, T.I. Murphy, D. Homan, D.K. Chantziantoniou, S. Khan, Retest reliability of medial frontal negativities during performance monitoring, Psychophysiol. 47 (2010) 260–270. [28] A. Weinberg, G. Hajcak, Longer term test–retest reliability of error-related brain activity, Psychophysiol. 48 (2011) 1420–1425. [29] S.M. Cassidy, I.H. Robertson, R.G. O’Connell, Retest reliability of event-related potentials: evidence from a variety of paradigms. Psychophysiol. 49 (2012) 659–664. [30] D. Hammerer, S.C. Li, M. Volkle, V. Muller, U. Lindenberger, A lifespan comparison of the reliability, test-retest stability, and signal-to-noise ratio of event-related potentials assessed during performance monitoring, Psychophysiol. 50 (2013) 111–123. [31] R. Huffmeijer, M.J. Bakermans-Kranenburg, L.R.A. Alink, M.H. van IJzendoorn, Reliability of event-related
potentials: The influence of number of trials and electrodes, Physiol. and Behav. 130 (2014) 14-22. [32] M. H. Hall, Ζ. K. Schulze, F. Rijsdijk, M. Picchioni, U. Ettinger, E. Bramon, R. Freedman, R. M. Murray, P. Sham, Heritability and Reliability of P300, P50 and Duration Mismatch Negativity, Behav Genet 36 (2006) 845–857. [33] A.J Bell, T.J. Sejnowski, An information-maximization approach to blind separation and blind deconvolution. Neural Comput. 7 (1995) 1129–1159. [34] A. Delorme A, S. Makeig, EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics, J Neurosci Methods 134 (2004) 9–21. [35] P.E. Shrout, J.L. Fleiss, Intraclass correlations: uses in assessing rater reliability, Psychol Bull. 86 (1979) 420– 428. [36] L. Portney, M. Watkins, M., Foundations of Clinical Research: Application to Practice, second ed., Prentice Hall, 1999. [37] J.C. Nunnally, I.H .Bernstein, Psychometric Theory, McGraw-Hill, New York, 1994.
Figure 1. Grand average of ERP wave at O2 electrode (Go/NoGo condition) in first and second session with the topography of the P1 peak latency, Y-axis – amplitude in µV, X-axis – time relative to onset of stimulus 1. Bar indicates the appearance of the stimulus. Black line correspond to session 1 while light grey line correspond to session 2.
Figure 2. Grand average of ERP wave at T6 electrode (NoGo condition) in first and second session with the topography of the P2 peak latency, Y-axis – amplitude in µV, X-axis – time relative to onset of stimulus 1. Bar indicates the appearance of the stimulus. Black line correspond to session 1 while light grey line correspond to session 2.
Figure 3. Grand average of ERP Go wave and IC P3 Go wave in first and second session, Y-axis – amplitude in µV, X-axis – time relative to onset of stimulus 2. Bar indicates the appearance of the stimulus. a: Grand average of ERP wave at Pz electrode with the topography of the P3 peak latency (Go condition), b: Topography and activation curve of IC P3 Go back-projected to Pz electrode. Black line correspond to session 1 while light grey line correspond to session 2.
Figure 4. Grand average of ERP NoGo wave and ICs P3 NoGo early and late in first and second session, Y-axis – amplitude in µV,
X-axis – time relative to onset of stimulus 2. Bar indicates the appearance of the stimulus. a: Grand average of ERP wave at Cz electrode with the topography of the P3 peak latency (NoGo condition), b: Topography and activation curve of IC P3 NoGo early back-projected to Cz electrode, c: Topography and activation curve of IC P3 NoGo late back-projected to Cz electrode. Black line correspond to session 1 while light grey line correspond to session 2.
Table 1 ERP Amplitudes: Means (1st and 2nd), differences in means (3rd ), test-retest reliability via ICC (4th and 5th) Mean 2nd session (std) ‘peak’ 13.6(6.28)
Mean Diff3 ‘peak’
P1 occipital
Mean 1st session (std) ‘peak’ 13.70 (7.5)
-0.1
Single ICC3 ‘peak’ 0.96***
N1 occipital
-3.34(4.89)
-3.37(4.79)
-0.03
0.86**
0.88**
P2 NoGo wave
8.66(5.35)
8.8(4.9)
-0.14
0.94***
0.9***
N2 NoGo wave
-4.02(4.37)
-4.65(3.23)
0.63
0.68***
0.71***
P3 Go wave
10.77 (4.49)
9.61(4.49)
1.16*
0.85***
0.78***
IC P3 Go wave
8.23(3.59)
7.25(3.84)
0.98
0.80***
0.76***
P3 NoGo wave
9.0(7.02)
6.64(6.71)
2.44**
0.81***
0.79***
IC P3 NoGo wave early
6.11(4.04)
4.55(4.12)
1.56**
0.77***
0.78***
IC P3 NoGo wave late
3.95(3.19)
3.32(2.46)
0.63
0.78***
0.78***
Amplitudes (μV)
measure
Single measure ICC3 ‘vFA’ 0.95***
Table 2 ERP Latencies: Means (1st and 2nd), differences in means (3rd ), test-retest reliability via ICC (4th and 5th) Mean 2nd session (std) ‘peak’ 131.83(19.86)
Mean Diff3 ‘peak’
P1 occipital
Mean 1st session (std) ‘peak’ 132.17(23.63)
N1 occipital
238.67(32.51)
P2 NoGo wave
0.34
Single measure ICC3 ‘peak’ 0.89***
Single ICC3 ‘vFA’ 0.86***
244(30.62)
-5.33
0.75***
0.92***
244.9(26.9)
249.5(27.2)
-4.6
0.90***
0.91***
N2 NoGo wave
253.17(29.0)
252.17(36.7)
1
0.7***
0.9***
P3 Go wave
300.3(26.19)
309(29.65)
-8.7
0.31
0.41*
IC P3 Go wave
298.33(27.96)
310(33.38)
-11.67*
0.72***
0.71***
P3 NoGo wave
358.83(39.76)
344.67(40.5)
14.16*
0.69***
0.77***
IC P3 NoGo wave early
329.67(31.8)
327(32.88)
2.67
0.83***
0.81***
IC P3 NoGo wave late
381.5(33.32)
381.5(34.7)
0
0.78***
0.84***
Latencies (ms)
3 * < 0.05, **<0.01, ***<0.001
measure
S0304-3940(16)30080-5 http://dx.doi.org/doi:10.1016/j.neulet.2016.02.012 NSL 31840
To appear in:
Neuroscience Letters
Received date: Revised date: Accepted date:
10-11-2015 2-2-2016 4-2-2016
Please cite this article as: Kyveli Kompatsiari, Gian Candrian, Andreas Mueller, Test-retest reliability of ERP components: A short-term replication of a visual Go/NoGo task in ADHD subjects, Neuroscience Letters http://dx.doi.org/10.1016/j.neulet.2016.02.012 This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Test-retest reliability of ERP components: A short-term replication of a visual Go/NoGo task in ADHD subjects
Kyveli Kompatsiaria, Gian Candriana, Andreas Muellera1
a
Brain and Trauma Foundation Grisons, Chur, Switzerland
1
Corresponding Author: Andreas Mueller, Brain and Trauma Foundation Grisons, Poststrasse 22, CH-7000 Chur, [email protected]
Highlights
This paper presents the results of a short-term test-retest reliability study of ERPs and Independent Components (ICs) in children with ADHD during a visual Go/NoGo paradigm.
Importantly, ICs which are directly linked to distinct cognitive processes, show comparable results with the raw ERPs.
In most cases, amplitudes show ‘good’ to ‘excellent’ test–retest reliability results, while latencies varied from a ‘poor’ to an ‘excellent’ level.
The obtained test-retest reliability results of the ADHD subjects are similar with previous studies on neurotypical samples.
Abstract Event-related potentials (ERPs) have been widely used to investigate brain functioning in children with Attention Deficit Hyperactivity Disorder (ADHD) in both research and diagnostic settings. To ensure the efficiency of ERP techniques in ADHD diagnosis and in longitudinal observational studies, the test-retest reliability of the affected population must be validated. Thus, the present article assesses the short-term testretest reliability of certain early and late ERPs (i.e. P1, N1, N2, P2, P3), as well as independent components (ICs) decomposed from the above mentioned ERPs (IC P3 Go, IC P3 NoGo early, IC P3 NoGo late) relevant to ADHD, through the Intraclass Correlation Coefficient (ICC). More specifically, we employ a cued visual Go/NoGo paradigm for recording ERPs from 22 children with ADHD (mean age 12.2), twice within 30 minutes. Amplitudes and latencies are calculated by the 'peak amplitude' method and by a variation of the fractional area. Results for amplitudes lie mostly within the 'good' and 'excellent' range for both measurement methods, while ICC for latencies is more variable ranging from 'poor' to ‘excellent’ results. Crucially, the ICs, which are associated with distinct functionally independent processes of the executive attentional
system have shown a comparable test-retest reliability with the raw ERPs. Our results are consistent with other reliability studies of neurotypical population in the literature, and as such, consist initial evidence that ERPs could be reliable neurophysiological markers for the ADHD population.
Keywords Test-retest reliability; Event-related potentials; Independent Components, ADHD; Visual Continuous Performance Task, Intraclass Correlation Coefficient
1. Introduction A variety of tasks have been used to elicit event-related potentials (ERPs) related to human sensory, motor, and cognitive functions. The non-invasiveness of the technique and the possibility of a direct measurement of neurocognitive functions with an excellent temporal resolution have made ERPs one of the most widely used methods in cognitive neuroscience research. Studies exploring various aspects of brain functioning associated with psychiatric disorders report ERP differences related to sensory and cognitive-processing [1]. ADHD is a prevalent developmental disorder, characterized by varying levels of inattention, hyperactivity, and impulsivity symptoms, where deviances have been reported for several sensory and cognitive ERP components [2]. Using the Go/NoGo paradigm, Smith et al. found clear differences in the early processing components (P1, N1, P2) of responses to Go and NoGo stimuli, which they interpreted as reflecting problems with sensory registration and identification of stimuli for children with ADHD [3]. Several studies have reported N2 (NoGo) and P3 waves (Go/NoGO) to be deviant in ADHD [4-9]. N2 and P3 NoGo waves have been associated with inhibition processes. Due to its late appearance, P3 has been further related to
evaluation of inhibitory processes and a voluntary decision to withhold the motor response [10-17]. The parietal P3 Go has been suggested to be similar to P3b elicited in oddball paradigms. P3b amplitude is said to reflect the allocation of attentional resources, and latency reflects classification speed. P3b is also argued to reflect a link between perceptual classification and response selection, i.e., monitoring whether the stimulus is classified correctly to the appropriate response [18-19]. Independent components (ICs) have also been studied in ADHD. In a study on the classification of ADHD and neurotypical individuals, Mueller et al., reported differences in the sensory components, as well as deviances in latencies and amplitudes of ICs derive from P3 Go component with a parietal distribution (IC P3 Go) and from P3 NoGo component (IC P3 NoGo early, IC P3 NoGo late) with a central and frontocentral distribution, respectively [9]. The IC P3 NoGo late has been associated with action monitoring [2021]. Recent research findings indicate a strong correlation between IC P3 NoGo early with energization, namely the ability to voluntarily invest attentional effort to optimize behavior for achieving a goal [21]. Additionally, recent studies show that IC P3 Go posterior, like P3b [18], reflects the process of concluding that a stimulus is classified correctly to the appropriate response, with the amplitude reflecting the degree of confidence in this conclusion [19]. Several studies have addressed the test-retest reliability of the ERP approach in a variety of cognitive tasks with the neurotypical population. The results range from ‘poor’ to ‘excellent’ within the session, over hours, days and years [17, 22-32]. The most frequently studied components are the P3b wave, error-related negativity (ERN), and Mismatch Negativity (MMN). In the majority of reliability studies, amplitudes generally show higher reliability than latencies2. Despite the fact that most of reliability research has insofar focused on young adults or on a specific age range, two studies have already provided with evidence that there are not systematic differences in test-retest reliabilities of ERP amplitudes between different age groups [30, 26]. Cassidy et al. mention ICC results ranging between 0.01 to 0.9 and covering a wide range of
2 For contradictory results see Brunner et al., Segalowitz and Barnes, Mei Hua Hall et al [17,22,32]
perceptual and cognitive components (P1, N1, N170, P3a, P3b, error-related negativity, error positivity, P400) [29]. Similar to the nature of our paradigm, Fallgatter et al. report ‘moderate’ to ‘excellent’ Intraclass Correlation Coefficient (ICC) results for P3 Go and P3 NoGo components using a CPT task twice [25]. Furthermore, a recent study addressing a 6-month replicate of a visual continuous Go/NoGo task (the same task used in the present study), reports excellent ICC for latencies and peaks of the P3 NoGo wave and two ICs decomposed from the P3 NoGo wave, the IC P3 NoGo early and the IC P3 NoGo late [17]. Studies generally use inconsistent stimuli, paradigms, test-retest measures, age groups, and between-session intervals, making it difficult to compare results. Importantly, to the best of our knowledge, there is a lack of test-retest reliability on the ADHD population. In this light, our study aims to serve as an initial step toward the validation of research and clinical efficiency of the ERP method on the ADHD population. Systematically investigating test-retest reliability in affected population could crucially inform both diagnostic and treatment practice. A short period of the test repetition was selected here for avoiding potential confounds due to exogenous factors, such as school and medication. Children with ADHD perform a visual continuous Go/NoGo task with a between-session interval of 30 minutes. Such a task targets crucial mechanisms in the affected population, such as sustained attention and inhibition. Regarding the specific components used to validate the reliability, all the above-mentioned components commonly used in ADHD research [2-19], are tested at a representative electrode with high activation of the component. More specifically, a number of earlier components are examined at the occipital electrode P1, N1 after the first stimulus of target relevant conditions (Go & NoGo conditions) and P2 at temporal electrode after second stimulus of NoGo condition. Additionally, the reliability of P3 Go component in the parietal electrode (Pz) after the second stimulus of Go condition and the independent component IC P3 Go with a parietal topography is calculated. Finally, we also test the N2 and P3 component at Cz electrode after the second stimulus of NoGo condition, and two ICs, namely IC P3 NoGo early and IC P3 NoGo late, decomposed from the raw ERP with central and fronto-central distributions, respectively.
2. Materials and methods 2. 1 Participants Our participants consist the first twenty five children participating in the 3-year study of Brain and Trauma Foundation, which focuses on the longitudinal investigation of biomarkers on ADHD subjects by measuring their brain activity over periods of six months. The test-retest recordings were conducted in August and September of 2015 with an interval of 30 minutes between sessions. All recordings were performed in morning hours, from 8:00AM - 14:00PM. The participants were referred to the study via advertisements to the medical doctors. They had been diagnosed with ADHD prior to their participation by an independent psychiatrist. DSM-IV criteria for ADHD were employed to confirm the diagnosis. Children were all diagnosed with either combined (6 children) or inattentive (19 children) subtypes of ADHD. Eight of the subjects were receiving medication, either Methylphenidate of 20mg or Concerta of 54mg, however, participants were required to refrain from taking medication during 24 hours before testing. Moreover, subjects suffering from a neurological disorder, severe comorbidities, or brain traumatic injury were not included in the study. The study was approved by the local ethics committee and written informed consent was obtained from the participants and their parents prior to their involvement. Subjects did not receive any compensation for their participation. 2.2 EEG recording The EEG was recorded using a NeuroAmp® x23 (BEE Medic GmbH), a PC-controlled 21-channel digital electroencephalographic system with a DC-coupling and 24-bit resolution. Electrodes were placed according to the International 10-20 system using an electrode cap with tin electrodes (Electro-cap International Inc.). Quantitative data was obtained using ERPrec software. The input signals, referenced to the linked earlobes, were filtered between 0.5 and 50 Hz and sampled at a rate of 250 Hz. Impedance was kept below 5 kOhm for all electrodes. Montage was changed to common average reference montage prior to data processing.
Blinking artefacts were corrected by zeroing the activation curves of individual independent components, acquired by application of ICA on raw EEG signal, which correspond to eye blinks. In addition, epochs of the filtered electroencephalogram with excessive amplitude (>120 μV) and/or excessive (threshold = z-score of 6) 0 to 3 Hz and 20 to 50 Hz band frequency activities were automatically excluded from further analysis. EEG was manually inspected to verify and exclude additional artefacts. Finally, trials with omission and commission errors were excluded. 2.3 Behavioral Task The performed task was a Visual Continuous Performance Task (VCPT) with 400 trials and 4 different conditions of equal probability appearing in a pseudorandomized order, and a duration of 20 minutes. Every trial consists of the presentation of a pair of images (100ms) with 1000ms inter-stimulus interval. The Go condition consists of an animal followed by a second animal (A-A). In this condition, the participant should press with the right hand the mouse as fast as possible. The NoGo condition consists of an animal followed by a plant (A-P), where the subjects should withhold their response. The third condition consists of two plants (P-P) and the fourth of a plant followed by an image (showing a human profession) together with an aberrant sound (P-H). The participants performed a preparatory practice phase to get familiarized with the task, ranging from 20 to 50 trials depending on their level of comprehension. In the middle of the task 5 minutes of rest was ensured for every subject. This study focused only on the trials consisting of target relevant stimuli, namely A-A and A-P. Participants were included only if they had more than 30 trials in each condition. Furthermore, only participants with more than 30 omission errors (not responding in go condition) were excluded as an indicator of excessively poor task engagement. In total 22 right-handed participants between 10 to 15 years old (Mean Age = 12.2, SD = 1.4, 4 female) remained for further analysis. 2.4 Decomposition of Collection of ERPs into Independent Components
When Independent Component Analysis (ICA) is applied to a collection of ERPs, a minimum amount of 100 individuals is required for a robust decomposition [20]. Having a limited number of participants, the ICA Infomax algorithm [33] was applied to a collection of individual ERPs of 185 (88 female) healthy subjects between 10-15 years old from HBi normative database, where the same behavioral task was used. ICA was performed using the EEGLAB Matlab toolbox [34]. The ICA input data was the two-dimensional 19-scalp-locations x 185-ERP-time-series matrix. The IC P3 Go, IC P3 NoGo early and IC P3 NoGo late components were extracted by applying ICA to the 1000 ms time interval after the second stimulus of the Go and No/Go trials respectively. The computed spatial filters were then applied to the individual ERPs. Under this analysis, The P3 NoGo wave could be decomposed into two independent components (IC P3 NoGo early and IC P3 NoGo late), while the P3 Go wave was decomposed in one independent component, IC P3 Go. More information about the features of the abovementioned ICs, i.e., latencies, topographies and functional meaning can be found in Kropotov et al., 2011 [20]. ICs were back-projected to the electrode with the highest activation of the component. 2.5 Measurement Measurements of latency and amplitude of raw ERP waves and ICs were performed with the 'peak amplitude' method by an ERP expert. As an additional objective method, a variation of the fractional area approach (vFA) is used for comparison reasons. More specifically, a latency window is defined for each extremum under analysis on the grand average curve. The window starts where the amplitude reaches the 50% amplitude value, between the extremum in question and its closest neighboring peak, and ends at the same amplitude value at the opposite side. As a following step, in this predefined latency window, the extremum value is selected on every individual curve and the area under/above the curve in this equally sized time window centered at the individual extremum is calculated.
2.6 Statistics The ICC was used as a test-retest reliability measure. ICC is an efficient method to measure test-retest reliability since it considers both the inter-subject and between-subjects variation. A two-way mixed effect model was employed with the setting of absolute agreement, showing to which degree the actual values are similar. The two-way mixed model was chosen because the error variance in measuring and scoring the ERPs is considered minimal while the error variance of the ERPs observations is regarded as unknown [35]. The reliability classification rates of Portney and Watkins was applied; namely, ICC below .50 is characterized as poor, moderate from .50 to .75, good from .75 to .90 and excellent when higher than .90 [36]. The means of the behavioral parameters and the ERP amplitudes and latencies between the two recordings were compared using two-tailed Paired-Sample T-Tests.
3. Results The mean epochs of A-A condition in the first session were 72 (±17) and 65 (±17) in the second, without showing a significant difference (t (21) = 0.93, p=0.36). The mean epochs of A-P condition were 76 (±14) in the first session and 72 (±15) in the second (t (21) = 1.5, p= 0.14). Furthermore, the average reaction time (first session: 415.11±68.5 vs second session: 420.48±83.3) does not differ significantly between the two measurements (t (21) = -1.29, p= 0.29). Finally, the average omission errors (first session: 7.4±7.48 vs second session: 8.27±6.22), also do not differ significantly between the two sessions (t (21) = -0.82, p= 0.42). The following tables 1, 2 show the statistical results of the ERP components between the two sessions for amplitudes and latencies respectively. More specifically, the first two columns illustrate the mean and standard deviation of the ERP parameters in the two recordings (only the peak method is used as a representative). The third column depicts the difference in the means (Mean Diff) between the two
measurements to investigate any systematic differences. The subsequent columns show the test-retest reliability results assessed with ICC for all the ERP components. The fourth represents the ‘peak’ method while the fifth the variation of the fractional area method, ‘vFA’. With regard to amplitudes, ADHD showed ‘good’ to ‘excellent’ reliability results in the majority of the components (Table 1). In contrast, latencies showed a higher variability, ranging from an ‘excellent’ to a ‘poor’ range. With respect to the ICs, reliability for amplitudes was slightly dropped compared to raw ERPs, but still remained within the ‘good’ range. Reliability regarding the latencies of the ICs is obviously larger than the raw ones. Using the vFA method, the reliability of the amplitudes showed a relatively small difference in comparison with the ‘peak’ method, while latencies showed better reliability on average. Moreover, regarding the variation of the reliability over time, a decline is clearly observed between the early and the later components. The means in the majority of the components did not differ significantly across sessions. However, the mean amplitudes of the raw P3 Go and NoGo waves, and the IC P3 NoGo early were significantly reduced from session 1 to session 2. Additionally, the mean latency was significantly increased in IC P3 Go and decreased in P3 NoGo. The grand average of the visual component at occipital electrode O2 for the two sessions and the topography of P1 peak latency are depicted in Figure 1. The grand average of the P2 NoGo wave at temporal electrode T6 and the topography of P2 peak latency are shown in Figure 2. The grand average of the P3 Go wave at Pz electrode and the topography of P3 peak latency are illustrated in Figure 3a. The P3 Go wave was decomposed in one IC, i.e., the IC P3 Go. The peak latency of IC P3 Go is around 300 ms after the 2nd stimulus (Go condition) and parietally distributed. The IC was back-projected to the Pz electrode where the highest activation of the component was situated. The topography and the activation curve of IC P3 Go are presented in Figure 3b for both sessions. The grand average of the P3 NoGo wave at Cz electrode and the topography of P3 peak latency is shown in Figure 4a. The P3 NoGo wave was decomposed in two ICs, i.e., the IC P3 NoGo early and IC P3 NoGo late. The IC P3 NoGo early is characterized by a positive deflection peaking around 330 ms after the onset of stimulus 2 and is centrally distributed. The IC P3 NoGo late shows
a positive peak around 380 ms post-stimulus 2 and has a central distribution as well. Both ICs were backprojected to the Cz electrode. The topographies and the activation curves of IC P3 NoGo early and IC NoGo late are presented in Figure 4b and 4c respectively.
Discussion The current study presents, to the best of our knowledge, the first short-term test-retest reliability results of several ERP components evoked during a Go/NoGo VCPT task, and relevant ICs derived from raw ERPs of Go/NoGo conditions, as tested on an ADHD sample. Amplitudes and latencies were extracted by the ‘peak amplitude’ method and by a variation of the fractional area. In total, in the major part of the components, subjects showed ‘good’ to ‘excellent’ reliability results concerning the amplitudes, while the latencies varied within the range of ‘poor’ to ‘excellent’ values. Our findings are in line with the majority of test-retest reliability studies and reviews, where both amplitudes and latencies were assessed [22-26,28,29]. Although it is difficult to compare reliability studies due to the variations in the tasks, reliability measures, and the test-retest interval, our study can be compared to some extent with three studies of reliability, Fallgatter et al. (2002), Brunner et al. (2013), and Cassidy et al. (2012) [25, 17, 29]. The similarity with the first two studies concern the nature of the performed task, the cognitive components evoked by this task and the ICs. Fallgatter et al. using a CPT task twice, report ‘good’ to ‘excellent’ ICC results for the peak amplitudes of P3 Go, P3 NoGo and ICs derived from these two conditions. Additionally, latencies show lower reliability values ranging from ‘good’ in NoGo condition to a not significant level in Go condition. These findings are in line with our findings where amplitudes lied in an identical range, whereas values for latencies were lower, with P3 Go latency showing also not significant results. However, Fallgatter et al. conducted a long-term reliability study for adult controls, using a different CPT task, and the model of ICC is not explicitly specified. In another long-term reliability study concerning young adult controls, Brunner et al., using the same VCPT task, the same way of ICA extraction and the same ICC measure, they report
good reliability (ICC > 0.75) for the amplitudes and excellent reliability (ICC > 0.9) for the latencies of ICs derived from NoGo condition. In our study, the reliability of the corresponding amplitudes lie also in the ‘good’ level, while reliability of latencies lie within a lower range compared to Brunner et al. study. In agreement with their work, in our study the area method had a greater positive impact on the reliability of the latencies. Concerning the earlier perceptual components, which are less affected by the task, we can compare our results with those found by Cassidy et al., where they state from moderate (latencies) to good (amplitudes) results using ICC in a visual oddball paradigm in P1 and N1 components (1-month inter-session interval) [29].Our study show higher values using both ‘peak’ and area methods, ranging from 0.75 to 0.96 for the early components. Importantly, in our study ICs show ‘good’ test-retest reliability results. Latencies of ICs are higher in comparison with raw ERPs probably due to their more concrete time decomposition. It is crucial to ensure a high test-retest reliability of ICs because each of them is linked to specific cognitive processes and would be beneficial to utilize them in a psychiatry context to reveal subject-specific traits (see Introduction part). In a number of cases, such as N2 NoGo peak and latency, P3 Go latency and P3 NoGo latency, a moderate to poor ICC is observed. In general, a drop in the reliability results is noticeable between early sensory and later components. Typically, the late components show higher variability than early ones because they are controlled more by endogenous factors than by the physical features of exogenous stimulation. According to Huffmeijer et al., at least 30 trials are required for early components while at least 60 trials, for later, broadly distributed components such as the P3 [31]. Later components reflecting more complex processes seem to require a higher number of trials for a larger reliability, which consists an intriguing part for children with ADHD. A systematic decline in amplitudes of later components is observed from the first to the second session. Certain components are significantly reduced, namely the amplitudes of the P3 Go, linked with allocation of attentional resources and stimulus classification monitoring, the P3 NoGo, related to the voluntary
decision to withhold a motor response, and IC P3 NoGo early, correlated with energization. One could argue that these components are generally linked with task engagement and conscious attentional effort and consequently, this observation might point to a slightly altered psychological state of the participants, for instance due to fatigue or adaptation effects. Although regular breaks were used to ensure the ease of the participants it might seem that a more extended retest time interval could alleviate this effect. This is the first study to explore the test-retest reliability of an ADHD sample on sensory and later ERP components and ICs decomposed from the raw ERPs. As stated by Nunnally and Bernstein [37], test-retest reliability of 0.80 is adequate for research experimental studies and 0.90 should be minimum for clinical purposes. Subsequently, according to our results most of the components could be safely used for research purposes, but not necessarily for clinical assessments. However, by addressing some limitations of the current study the stability of ERP components could be enhanced. For instance, the effect of the number of the averaged trials on the reliability of components in subjects with ADHD remains an important issue to be studied in future studies. Furthermore, although we think it is crucial to test the reliability of the affected population with a short repetition time in order to avoid long term interventional effects, considering a longer time period between tests might prove beneficial. In the latter case, side-effects from potential fatigue would be expected to be alleviated in a greater extent. Moreover, it would be highly informative to compare reliability results of an extended ADHD sample to neurotypicals, as well as to consider additional age groups. Concluding, we feel that more extensive research in this direction would be of great importance for encouraging the applicability of ERPs on clinical and research settings both on a diagnostic and treatment level, as for instance with regard to testing competing treatment strategies.
Conclusions To the best of our knowledge, this is the first study to examine the test-retest reliability of raw ERPs and independent components (ICs) on ADHD children. In particular, amplitudes showed ‘good’ to ‘excellent’ reliability results, while latencies varied from ‘poor’ to ‘excellent’ values. More importantly, ICs, crucial in psychiatry due to their association with different, functionally independent processes of the executive attentional system have shown a comparable test-retest reliability with the raw ERPs. Additionally, components of the ADHD subjects showed similar reliability results with previous studies on neurotypical samples. This study can serve as a first step toward the validation of ERPs and ICs as reliable biomarkers in ADHD. Our positive results indicate that such metrics, after further assessment, could be potentially employed by researchers and clinicians as an assistive tool in future diagnostic and treatment practice.
Conflict of interest statement Authors state no conflict of interest.
Acknowledgments The study was funded by Brain and Trauma Foundation. Additionally, the authors thank dr. Bernhard Wandernoth and Michael Schweikert for technical support. Finally, we are grateful to Gian Ruschetti for assistance in data acquisition and Dimitrios Bolis for scientific support.
References [1] J.D. Kropotov, Quantitative EEG, Event-Related Potentials and Neurotherapy, first ed. Academic Press, San Diego, 2009. [2] R.J. Barry, S.J. Johnstone, A.R. Clarke, A review of electrophysiology in attention-deficit/hyperactivity disorder II: Event-related potentials, Clin Neurophysiol. 114 (2003) 184-198. [3] J.L. Smith, S.J. Johnstone, R.J. Barry, Inhibitory processing during the Go/NoGo task: an ERP analysis of children with attention-deficit/hyperactivity disorder, Clin Neurophysiol. 115 (2004) 1320-1331. [4] M. Spronk, L.M. Jonkman, C. Kemner, Response inhibition and attention processing in 5- to 7-year-old children with and without symptoms of ADHD: an ERP study, Clin Neurophysiol. 119 (2008) 2738–2752. [5] M. Doehnert, D. Brandeis, K. Imhof, R. Drechsler, H.C. Steinhausen, Mapping attention-deficit/hyperactivity disorder from childhood to adolescence: no neurophysiologic evidence for a developmental lag of attention but some for inhibition, Biol Psychiatry. 7 (2010) 608–616. [6] D. Brandeis, T. Banaschewski, L. Baving, et al., Multicenter P300 brain mapping of impaired attention to cues in hyperkinetic children, J Am Acad Child Adolesc Psychiatry 41 (2002) 990–998. [7] J.R. Wiersema, H. Roeyers, ERP correlates of effortful control in children with varying levels of ADHD symptoms, J Abnorm Child Psychol. 37 (2009) 327–336. [8] S.J Johnstone, R.J Barry, A.R Clarke, Ten years on: a follow-up review of ERP research in attentiondeficit/hyperactivity disorder, Clin Neurophysiol. 124 (2013) 644–657. [9] A. Mueller, G. Candrian, J.D Kropotov, V.A Ponomarev, G.M. Baschera, Classification of ADHD patients on the basis of independent ERP components using a machine learning system, Nonlinear Biomed Phys. 4 (2010) S1. [10] A.J Fallgatter, W.K Strik, The NoGo-anteriorization as a neurophysiological standard-index for cognitive response control, Int J Psychophysiol. 32 (1999) 233-238. [11] A. Kok, J.R. Ramautar, M.B. De Ruiter, G.P.H. Band, K.R. Ridderinkhof, ERP components associated with successful and unsuccessful stopping in a stop-signal task, Psychophysiol. 41 (2004) 9–20.
[12] K.J. Bruin, A.A. Wijers, A.S. van Staveren, Response priming in a go/nogo task:do we have to explain the go/nogo N2 effect in terms of response activation instead of inhibition?, Clin Neurophysiol. 112 (2001) 1660–1671. [13] M. Liotti, S.R. Pliszka, R. Perez, D. Kothmann,, M.G Woldorff, Abnormal brain activity related to performance monitoring and error detection in children with children with ADHD, Cortex 41 (2005) 377–388. [14] M. Schmajuk, M. Liotti, L. Busse, M.G. Woldorff, Electrophysiological activity underlying inhibitory control processes in normal adults. Neuropsychologia 44 (2006) 384–395. [15] S. van Gaal, V.A.F. Lamme, J.J Fahrenfort, K.R. Ridderinkhof, Dissociable brain mechanisms underlying the conscious and unconscious control of behavior, J Cogn Neurosci. 23 (2011) 91–105. [16] P.D. Gajewski, M. Falkenstein, Effects of task complexity on ERP components in Go/Nogo tasks, Int J Psychophysiol. 87 (2013) 273–278. [17] J. F. Brunner, T.I. Hansen, O. Alexander, S. Toril, H. Asta, J. Kropotov, Long-term test-retest reliability of the P3 NoGo wave and two independent components decomposed from the P3 NoGo wave in a visual Go/NoGo task, Int J Psychophysiol. 89 (2013) 106–114. [18] R. Verleger, P. Jaskowski, & E. Wascher, Evidence for an integrative role of P3b in linking reaction to perception. J Psychophysiol. 19 (2005) 165–181. [19] I.E. Aasen, J.F. Brunner, Modulation of ERP components by task instructions in a cued 2 go/no-go task, Psychophysiol. (2015), in press. [20] J. D. Kropotov, V.A. Ponomarev, S. Hollup, A. Mueller, A., Dissociating action inhibition, conflict monitoring and sensory mismatch into independent components of event related potentials in GO/NOGO task. NeuroImage 57 (2011) 565–575. [21] J.F. Brunner, A. Olsen, I.E. Aasen, G.C. Løhaugen, A.K. Håberg, J. Kropotov Neuropsychological parameters indexing executive processes are associated with independent components of ERPs, Neuropsychologia 66 (2015) 144– 156. [22]S.J. Segalowitz, K.L. Barnes, The reliability of ERP components in the auditory oddball paradigm. Psychophysiol. 30 (1993) 451–459. [23] M. Tervaniemi, A. Lehtokoski, J. Sinkkonen, J. Virtanen, R.J. Ilmoniemi, R. Näätänen, Test-retest reliability of mismatch negativity for duration, frequency and intensity changes, Clin. Neurophysiol. 110 (1999) 1388-1393.
[24] C.A. Sandman, J.V. Patterson, The auditory event-related potential is a stable and reliable measure in elderly subjects over a 3 year period, Clin Neurophysiol. 111 (2000) 1427–1437. [25] A.J. Fallgatter, D.R. Aranda, A.J. Bartsch, M. J. Herrmann, Long-term Reliability of Electrophysiologic Response Control Parameters, Clin Neurophysiol. 19 (2002) 61–66. [26] K.B. Walhovd, A.M Fjell, One-year test–retest reliability of auditory ERPs in young and old adults. Int J Psychophysiol. 46 (2002) 29–40. [27] S.J. Segalowitz, D.L. Santesso, T.I. Murphy, D. Homan, D.K. Chantziantoniou, S. Khan, Retest reliability of medial frontal negativities during performance monitoring, Psychophysiol. 47 (2010) 260–270. [28] A. Weinberg, G. Hajcak, Longer term test–retest reliability of error-related brain activity, Psychophysiol. 48 (2011) 1420–1425. [29] S.M. Cassidy, I.H. Robertson, R.G. O’Connell, Retest reliability of event-related potentials: evidence from a variety of paradigms. Psychophysiol. 49 (2012) 659–664. [30] D. Hammerer, S.C. Li, M. Volkle, V. Muller, U. Lindenberger, A lifespan comparison of the reliability, test-retest stability, and signal-to-noise ratio of event-related potentials assessed during performance monitoring, Psychophysiol. 50 (2013) 111–123. [31] R. Huffmeijer, M.J. Bakermans-Kranenburg, L.R.A. Alink, M.H. van IJzendoorn, Reliability of event-related
potentials: The influence of number of trials and electrodes, Physiol. and Behav. 130 (2014) 14-22. [32] M. H. Hall, Ζ. K. Schulze, F. Rijsdijk, M. Picchioni, U. Ettinger, E. Bramon, R. Freedman, R. M. Murray, P. Sham, Heritability and Reliability of P300, P50 and Duration Mismatch Negativity, Behav Genet 36 (2006) 845–857. [33] A.J Bell, T.J. Sejnowski, An information-maximization approach to blind separation and blind deconvolution. Neural Comput. 7 (1995) 1129–1159. [34] A. Delorme A, S. Makeig, EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics, J Neurosci Methods 134 (2004) 9–21. [35] P.E. Shrout, J.L. Fleiss, Intraclass correlations: uses in assessing rater reliability, Psychol Bull. 86 (1979) 420– 428. [36] L. Portney, M. Watkins, M., Foundations of Clinical Research: Application to Practice, second ed., Prentice Hall, 1999. [37] J.C. Nunnally, I.H .Bernstein, Psychometric Theory, McGraw-Hill, New York, 1994.
Figure 1. Grand average of ERP wave at O2 electrode (Go/NoGo condition) in first and second session with the topography of the P1 peak latency, Y-axis – amplitude in µV, X-axis – time relative to onset of stimulus 1. Bar indicates the appearance of the stimulus. Black line correspond to session 1 while light grey line correspond to session 2.
Figure 2. Grand average of ERP wave at T6 electrode (NoGo condition) in first and second session with the topography of the P2 peak latency, Y-axis – amplitude in µV, X-axis – time relative to onset of stimulus 1. Bar indicates the appearance of the stimulus. Black line correspond to session 1 while light grey line correspond to session 2.
Figure 3. Grand average of ERP Go wave and IC P3 Go wave in first and second session, Y-axis – amplitude in µV, X-axis – time relative to onset of stimulus 2. Bar indicates the appearance of the stimulus. a: Grand average of ERP wave at Pz electrode with the topography of the P3 peak latency (Go condition), b: Topography and activation curve of IC P3 Go back-projected to Pz electrode. Black line correspond to session 1 while light grey line correspond to session 2.
Figure 4. Grand average of ERP NoGo wave and ICs P3 NoGo early and late in first and second session, Y-axis – amplitude in µV,
X-axis – time relative to onset of stimulus 2. Bar indicates the appearance of the stimulus. a: Grand average of ERP wave at Cz electrode with the topography of the P3 peak latency (NoGo condition), b: Topography and activation curve of IC P3 NoGo early back-projected to Cz electrode, c: Topography and activation curve of IC P3 NoGo late back-projected to Cz electrode. Black line correspond to session 1 while light grey line correspond to session 2.
Table 1 ERP Amplitudes: Means (1st and 2nd), differences in means (3rd ), test-retest reliability via ICC (4th and 5th) Mean 2nd session (std) ‘peak’ 13.6(6.28)
Mean Diff3 ‘peak’
P1 occipital
Mean 1st session (std) ‘peak’ 13.70 (7.5)
-0.1
Single ICC3 ‘peak’ 0.96***
N1 occipital
-3.34(4.89)
-3.37(4.79)
-0.03
0.86**
0.88**
P2 NoGo wave
8.66(5.35)
8.8(4.9)
-0.14
0.94***
0.9***
N2 NoGo wave
-4.02(4.37)
-4.65(3.23)
0.63
0.68***
0.71***
P3 Go wave
10.77 (4.49)
9.61(4.49)
1.16*
0.85***
0.78***
IC P3 Go wave
8.23(3.59)
7.25(3.84)
0.98
0.80***
0.76***
P3 NoGo wave
9.0(7.02)
6.64(6.71)
2.44**
0.81***
0.79***
IC P3 NoGo wave early
6.11(4.04)
4.55(4.12)
1.56**
0.77***
0.78***
IC P3 NoGo wave late
3.95(3.19)
3.32(2.46)
0.63
0.78***
0.78***
Amplitudes (μV)
measure
Single measure ICC3 ‘vFA’ 0.95***
Table 2 ERP Latencies: Means (1st and 2nd), differences in means (3rd ), test-retest reliability via ICC (4th and 5th) Mean 2nd session (std) ‘peak’ 131.83(19.86)
Mean Diff3 ‘peak’
P1 occipital
Mean 1st session (std) ‘peak’ 132.17(23.63)
N1 occipital
238.67(32.51)
P2 NoGo wave
0.34
Single measure ICC3 ‘peak’ 0.89***
Single ICC3 ‘vFA’ 0.86***
244(30.62)
-5.33
0.75***
0.92***
244.9(26.9)
249.5(27.2)
-4.6
0.90***
0.91***
N2 NoGo wave
253.17(29.0)
252.17(36.7)
1
0.7***
0.9***
P3 Go wave
300.3(26.19)
309(29.65)
-8.7
0.31
0.41*
IC P3 Go wave
298.33(27.96)
310(33.38)
-11.67*
0.72***
0.71***
P3 NoGo wave
358.83(39.76)
344.67(40.5)
14.16*
0.69***
0.77***
IC P3 NoGo wave early
329.67(31.8)
327(32.88)
2.67
0.83***
0.81***
IC P3 NoGo wave late
381.5(33.32)
381.5(34.7)
0
0.78***
0.84***
Latencies (ms)
3 * < 0.05, **<0.01, ***<0.001
measure