Human EEG reveals distinct neural correlates of power and precision grasping types

Accepted Manuscript Human EEG reveals distinct neural correlates of power and precision grasping types Iñaki Iturrate, Ricardo Chavarriaga, Michael Pereira, Huaijian Zhang, Tiffany Corbet, Robert Leeb, José del R. Millán PII:

S1053-8119(18)30670-0

DOI:

10.1016/j.neuroimage.2018.07.055

Reference:

YNIMG 15146

To appear in:

NeuroImage

Received Date: 22 January 2018 Revised Date:

11 June 2018

Accepted Date: 23 July 2018

Please cite this article as: Iturrate, Iñ., Chavarriaga, R., Pereira, M., Zhang, H., Corbet, T., Leeb, R., Millán, José.del.R., Human EEG reveals distinct neural correlates of power and precision grasping types, NeuroImage (2018), doi: 10.1016/j.neuroimage.2018.07.055. This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

ACCEPTED MANUSCRIPT

Human EEG reveals distinct neural correlates of power and precision grasping types

for Neuroprosthetics (CNP), Defitech Chair in Brain-Machine Interface (CNBI). École Polytechnique Fédérale de Lausanne (EPFL), Switzerland. Abbreviated title: EEG neural correlates of grasping types *Corresponding author: [email protected]

SC

1Center

RI PT

Iñaki Iturrate*, Ricardo Chavarriaga, Michael Pereira, Huaijian Zhang, Tiffany Corbet, Robert Leeb and José del R. Millán

Abstract

AC C

EP

TE D

M AN U

Hand grasping is a sophisticated motor task that has received much attention by the neuroscientific community, which demonstrated how grasping activates a network involving parietal, pre-motor and motor cortices using fMRI, ECoG, LFPs and spiking activity. Yet, there is a need for a more precise spatiotemporal analysis as it is still unclear how these brain activations over large cortical areas evolve at the sub-second level. In this study, we recorded ten human participants (1 female) performing visually-guided, self-paced reaching and grasping with precision or power grips. Following the results, we demonstrate the existence of neural correlates of grasping from broadband EEG in self-paced conditions and show how neural correlates of precision and power grasps differentially evolve as grasps unfold. 100 ms before the grasp is secured, bilateral parietal regions showed increasingly differential patterns. Afterwards, sustained differences between both grasps occurred over the bilateral motor and parietal regions, and medial pre-frontal cortex. Furthermore, these differences were sufficiently discriminable to allow singletrial decoding with 70% decoding performance. Functional connectivity revealed differences at the network level between grasps in fronto-parietal networks, in terms of upper-alpha cortical oscillatory power with a strong involvement of ipsilateral hemisphere. Our results supported the existence of fronto-parietal recurrent feedback loops, with stronger interactions for precision grips due to the finer motor control required for this grasping type. Keywords: Brain-machine interfaces, Grasping, Neural correlates, EEG

Preprint submitted to Neuroimage

July 25, 2018

ACCEPTED MANUSCRIPT

1. Introduction

AC C

EP

TE D

M AN U

SC

RI PT

Hand grasping is essential for activities of daily life. Nonetheless, such a seemingly easy task for humans involves extended brain networks which, although widely studied [1], are still not fully understood. Electrophysiological studies have been performed with invasive recordings, using single-unit activity in non-human primates [2] and in humans [3], as well as with local field potentials (LFPs) [4] and electrocorticography (ECoG) in humans [5, 6]. Studies have also shed light on the brain regions involved in grasping using functional resonance imaging (fMRI) [7, 8, 9]. It is now widely believed that grasping activates a complex network involving parietal regions (anterior intraparietal sulcus), premotor regions (ventral premotor cortex), and primary motor cortex [10]. Among the many studies on grasping, the differences between the two most common grasping types (power and precision grips) has received much attention [11, 12, 13]. Despite these findings, recent works have pointed out the necessity of more finely grained analyses on the spatio-temporal evolution of reachingand-grasping neural correlates [14, 15]. Non-invasive electroencephalography (EEG) offers an additional alternative to study grasping, as it provides ac- curate temporal resolution of neural activity over large brain regions. Several works have used EEG to study the event-related potentials associated to grasping during cued scenarios [16]. Yet, there is still a lack of EEG-based studies of grasping under ecological conditions, i.e. when grasping is performed under self-paced conditions in situations resembling activities of daily living. Besides basic neuroscience, there is increasing interest in predicting grasping at single-trial level for brain-machine interfacing (BMI) and its use on upper-limb neuroprostheses. On one hand, much attention has been devoted to the decoding of movement onset from EEG (e.g. see [17, 18]); or grasping onset from ECoG [6] and EEG in one of our previous studies [19]. On the other hand, decoding of grasping types has been mainly studied using invasive recordings such as spiking rates [3], ECoG [20] and fMRI [21], but EEG-based studies have been very limited. Recently Agashe et al. have addressed EEG prediction of grasping types by the decoding of joint angles and movement synergies [22, 23], an approach similarly followed by Schwarz et al [24], and Jochumsen et al. [25]. These works based their analysis on the use of very low frequencies (<5 Hz). Thus, there is still a lack of basic studies investigating the existence of broadband EEG correlates of complex task-oriented movements such as grasping, both in terms of spatio-temporal brain areas and networks involved. In this study, ten human participants (one female) performed self-paced

ACCEPTED MANUSCRIPT

AC C

EP

TE D

M AN U

SC

RI PT

visually-guided reaching and grasping with precision or power grips (Figure 1). In our experimental design, we aimed at resembling an ecological condition where subjects decide when (self-paced task) and how to perform a grasping (self-chosen type of grasping). As such, we considered grasping as a high-level combination of its associated kinetic and kinematic parameters, mainly: direction, force, joints rotation, position and speed; and evaluated it as a whole rather than each parameter separately. We detail differences at the sub-second level of activated brain regions and networks over the whole scalp between the two grasping types over a broadband spectrum EEG. Results revealed a complex differentiable pattern time-locked to the moment the grasping is secured over the object: During the early pre-shaping phase, EEG analysis revealed that precision and power grasps differed in the activation over contra-lateral motor and parietal cortices, a pattern consistent with previous studies using fMRI [11, 21]. As the grasping unfolded, a sustained difference between both grasping types was present over contralateral pre-frontal and ipsilateral motor and parietal regions. We report network-level differences between precision and power grasps. Modulations in lagged functional connectivity in the upper-alpha band were observed bilaterally over fronto-parietal networks; supporting the existence of frontoparietal recurrent feedback loops [12, 15, 26]. Observed modulations were in accordance to current opinions proposing that alpha oscillations are enhanced in tasks requiring more top-down control [27]: Precision grips required finer motor control than power grips, eliciting stronger modulations in the upper alpha frequency band. Notably, using a multi-dimensional decoding algorithm, we were able to exploit these temporal differences to decode grasp types in single trials with a decoding performance of 70%, opening new avenues for non-invasively controlled neuroprostheses for assistive and neurorehabilitation scenarios.

ACCEPTED MANUSCRIPT

2. Materials and methods

M AN U

SC

RI PT

As briefly mentioned in the introduction, most works studying grasping on EEG have focused mostly on the decoding of grasping types using slow-cortical potentials [22, 25, 24], while advances at a more basic neuroimaging level are lacking. In this work, we aimed at providing more evidence on the EEG neural correlates of grasping from a broadband EEG. More specifically, the main purpose of this study is to provide an exploratory characterization and analysis of precision vs power grasps using EEG from three complementary perspectives while making results comparable with previous works: First, a spatio- and spectro-temporal analysis of the evolution of two types of grasping, together with their decoding in single trials to assess how consistent the identified correlates are at a single-trial level; second a source localization analysis to better identify the loci of these patterns, allowing a comparison with previous fMRI studies (e.g. [11, 21]); and third, a connectivity analysis to provide an insight on the network processes that mediate grasping. In the remaining of this section, we discuss the experimental setup as well as the specifics of the analyses performed.

AC C

EP

TE D

2.1. Experimental setup We designed an experimental protocol to induce self-paced reach-to-grasp movements (similar to [6, 20]), executed without any external cue in order to better simulate hand motor tasks during activities of daily living (ADL). During the experiment, participants were comfortably seated on a chair facing an object that they had to grasp. Ten right-handed subjects (1 female, mean age 27±4 years) participated in the recordings. The study was approved by the local ethical committee, and subjects gave their written permission via a signed consent form. For each subject, recordings were performed in a single session that lasted approximately two hours including EEG setup and removal. Each trial began with subjects resting their dominant (right) hand on a button on the table. At their own pace, but after an inter-trial interval of at least two seconds, they reached and then grasped an object placed in front of them (see Figure 1). The object could be grasped in two different ways: power and precision grip. The type of grasping was freely chosen by the subject at each trial. The reaching-and-grasping phase lasted 1015±222 ms on average for all subjects. After grasping the object, subjects were instructed to lift it, place it back to its original position and move their hand back to the rest position (on top of the button). The lifting phase lasted 2040±596 ms on average for all subjects. Every 100 trials, the object was repositioned to a different place in order to avoid any laterality confounds. Four different placements were

ACCEPTED MANUSCRIPT

RI PT

chosen: the initial position, and 10 centimeters to the left, right and straight further away from the initial position. The distance from the resting position to the object ranged from approximately 40 to 60 cm depending on the object placement. Subjects were asked to restrain eye movements or blinks during reaching and grasping, and to fixate their gaze to a cross located behind the object on the wall, at one meter of distance. 400 trials per subject were recorded, and those where the inter-trial phase lasted less than two seconds were removed from the analysis. This led to an average per subject of 185±12 and 193±10 trials for precision and power grips respectively.

AC C

EP

TE D

M AN U

SC

2.2. Data collection and pre-processing EEG, EMG and EOG were recorded at 2048 Hz using a Biosemi system (Amsterdam, the Netherlands) with 64 active electrodes, equally distributed over the scalp following the 10/10 international system. EEG data were recorded from DC to 1024 Hz, and then offline bandpass filtered between 1 and 40 Hz (zero-phase Butterworth 4th order), downsampled to 256 Hz, and rereferenced to a common average (CAR). Three monopolar electrooculography (EOG) electrodes were placed above the nasion, and on the outer canthi of the left and right eyes, and were bandpass filtered as the EEG. Horizontal EOG was defined as the difference between the signals from the two outer canthi electrodes, and vertical EOG as the mean of the signals from the two outer canthi electrodes minus the signal from the nasion electrode. Horizontal and vertical EOG were then used to remove possible associated artifacts on the EEG using a regression algorithm [28]. Electromyographic (EMG) activity was recorded from a bipolar channel placed over the first dorsal interosseous (1DI) muscle of the grasping hand. EMG signal was then bandpass filtered between 30 and 100 Hz (zero-phase Butterworth 4th order) rectified and then smoothed with a low pass filter at 5 Hz. Both the rest position and the object used during the experiment were sensorized via three buttons (one for resting and one per type of grasping, see Figure 1a). The state of the buttons was recorded in parallel and at the same frequency to the electrophysiological activity in order to extract the onsets of the events of interest. This allowed recording data with a precise time synchronization between EEG-EMG signals and reaching/grasping onsets. Recorded data were then epoched to the onset of the grasp defined as the time when the subjects pressed the trigger to grasp. Epoched data were analyzed surrounding the grasping onset (t=0 ms).

2.3. Electrooculography analysis We evaluated the effect of possible EOG artifactual components on the EEG. Supplementary Figure S1 shows the Pearson correlation coefficients

ACCEPTED MANUSCRIPT

RI PT

before and after EOG removal by regression. As can be seen, EOG -especially vertical- mostly affected frontal areas, whereas central and parietal areas were much less affected by from such artifacts. Notwithstanding, both horizontal and vertical EOG did not affect the EEG after EOG removal by regression [28], with no significant correlation found for any of the EEG electrodes (r < 0.05, p > 0.5).

M AN U

SC

2.4. Data analysis In order to evaluate the existence of neural correlates of grasping, we performed an exploratory analysis in both temporal and spectral domains. Regarding the temporal domain, we were interested in differences at the EEG level before the grasping was secured. As such, we epoched the data using data prior to the grasping onset within a time window of [-500, 0] ms. While our analysis will focus mostly on this time window representing the grasping preshaping, we also show for the sake of representation the evolution after the grasping is secured, i.e. during the lifting phase. Thus, the temporal and spectral analysis is represented within the time window of [-500, 500] ms. We assessed then the existence of discriminant information (power vs precision grips) in the broadband EEG: delta (1-4 Hz), theta (5-8 Hz), lower alpha (8-10 Hz), upper alpha (10-12 Hz), lower beta (13-18 Hz) upper beta (1830 Hz) and low gamma (30-40 Hz) frequency bands.

AC C

EP

TE D

2.4.1. Temporal analysis Following previous findings on grasping correlates using ECoG and LFP signals [4, 20], we analyzed the time evolution of the grasping correlates. Grand-averaged signals (across trials and subjects) were computed for all EEG channels for both grasping conditions (precision and power), for the time window [-500, 500] ms. Bonferroni corrected paired t-tests were applied at the population level to evaluate the statistical significance on the averaged signals. Temporal discriminability between precision and power grips and across all channels was evaluated using the squared point biserial correlation between labels (i.e. type of grasping) and the temporal features, namely the r2 index. To assess the localization of the differences in temporal evolution between the two grasp types, we used sLoreta to estimate intracortical sources [29]. sLoreta inverse solution was calculated over the averaged temporal evolution of each subject, leading to one estimation per time point and subject. In order to reduce the inter-subject variability, voxel activations of each subject were normalized with respect to the maximum voxel value. Non-parametric statistical tests were run at the population level to locate significant temporal voxel differences between the two grasping types. A non-parametric paired

ACCEPTED MANUSCRIPT

RI PT

permutation test (5000 permutations) was computed by randomly shuffling the labels at each iteration and estimating the empirical probability distribution of t-values via paired t-tests. In order to correct for multiple comparisons, we computed the distribution of maximum and minimum t-values at each permutation, and used a 95% confidence interval to obtain the p-values. Note that this method corrects for multiple comparison type I errors [30].

AC C

EP

TE D

M AN U

SC

2.4.2. Spectral analysis Event-related spectral perturbations (ERSP) were computed to identify pairs of frequency-time components that carried discriminant information in the signal within the time window [-500, 500] ms for every frequency of interest [31]. We limited the analysis to a subset of electrodes covering contra-lateral and ipsi-lateral pre-motor, motor, and parietal regions (FC, C, CP, and P line of electrodes). Power spectral density of the epoched signal was computed via a sliding Hanning window of 500 milliseconds with an overlap of 99% and averaged across conditions and subjects, and baselined to a rest period defined as the window [-1000, -750] ms before movement onset. Statistical tests were executed at the population level to discern the differences between the two grasping conditions. A non-parametric test (5000 permutations) corrected for multiple comparisons was performed to evaluate statistical differences. Similarly to the temporal analysis, we also assessed the intracortical localization of the differences in spectral power using sLoreta [29]. First, epochs extracted within the window [-500, 0] ms were used to compute their cross spectrum for the frequency bands of interest. sLoreta inverse solution was calculated from the cross-spectrum results, leading to one estimation per frequency band and subject. In order to reduce the inter-subject variability, voxel activations of each subject were normalized with respect to the maximum voxel value. Non-parametric statistical tests were run at the population level to locate significant spectral voxel differences between the two grasping types and evaluated using permutation tests as in the temporal and ERSP analyses.

ACCEPTED MANUSCRIPT

2.4.3. Single-trial classification

EP

ℎ

TE D

M AN U

SC

RI PT

We evaluated the decoding capabilities of precision vs power grasps using single-trial classification based on the temporal evolution of bandpass-filtered EEG signals. To this end, we firstly performed a grid-search on the low and high cutoff frequencies for the bandpass filters. We then bandpass filtered the raw EEG based on this frequency band using a non-causal, 4th order filter, and downsampled the signal to 64 Hz. For each epoch (i.e. trial), we extracted the temporally filtered EEG values from 15 centro-parietal channels (C3, C1, Cz, C2, C4, CP3, CP1, CPz, CP2, CP4, P3, P1, Pz, P2, P4) within a time window of [-500, 0] ms, and concatenated within a feature vector of 480 features. This process led to a matrix of size features x trials. The resulting features were normalized within the range [0, 1]. Finally, the features' dimensionality was reduced by retaining 95% of the variance explained by their principal components (PCA). This process led to an average of 46±7 features. For more details on the feature extraction method, please refer to the block diagram presented in Supplementary Figure S2 and to [32]. A shrinkage linear discriminant (shrinkage LDA) [33] was used as classification method, with the regularization parameter empirically fixed to λ=0.1. We used 10-fold cross-validation to evaluate the performance of our decoder on unseen data. All hyperparameters, including normalization and PCA coefficients, were extracted from the training set of each fold. We evaluated the decoding performance using the class-wise accuracy, or hit rate, of precision (power) grasps, defined as the sum of correctly decoded precision (power) grasps divided by the total number of precision (resp. power) grasps:

AC C

ℎ

=

#



#

=



#

#

(1)







(2)

The performances were then averaged across folds. In order to test whether this performance was significantly above chance, we estimated the chance level as the 95% confidence interval of the inverse of the binomial distribution based on the number of trials available (200) and uniform priors (0.5), leading to a value of 55.72%. Additionally, we evaluated the decoding capacity of each of the 64 individual channels recorded. To this end, we repeated the same analysis but using a single channel each time, and we report the topographic interpolation

ACCEPTED MANUSCRIPT

of this decoding. As a further evaluation, we also analyzed the decoding capabilities when including information using information before and after the grasping onset (i.e. including the lifting phase), by using the window [-500,500] ms.

3. Results 3.1. Spatio-temporal analysis

M AN U

SC

RI PT

2.4.4. Functional connectivity analysis We performed a functional brain connectivity analysis estimated using lagged phase synchronization (LPS) at the source space [34], which does not suffer for volume conduction. For this analysis, 6 regions of interest (ROI) were defined: bilateral motor cortex (BA4), bilateral pre-motor cortex (BA6), and bilateral parietal cortices (BA7 and BA40), following previous studies on the regions activated during grasping [1]. Results were computed using sliding windows of 500 ms width (i.e., using data from the past 500 ms), with steps of 100 ms, from -500 to 0 ms, where 0 ms indicates the onset of grasping. Grouplevel analyses were performed for the frequency bands of interest. Similarly to the source space temporal and spectral analysis, p-values were estimated using non-parametric permutation tests.

AC C

EP

TE D

We first evaluated the temporal evolution of the EEG signals (Butterworth, 4th order bandpass filtered [1-40] Hz) obtained during power and precision grips, similarly to [4, 20]. We evaluated the differences in the EEG activity between precision versus power grasps by a discriminability index between both conditions (r2), see Figure 2a. From these results, three main regions of activation can be discerned prior to the grasping onset: contra and ipsilateral centro-parietal areas and a midline pre-frontal area. During the early preshaping phase (at roughly 400 ms before the grasping is secured, see Supplementary Figure S3), and as the pre-shaping of the hand unfolds prior to the grasp bilateral parietal regions showed increasingly differential patterns. As the grasping time approaches (from 100 to 50 ms before the onset), a sustained difference between both grasps occurred over the bilateral motor and parietal regions, and medial pre-frontal cortex. Once the grasping is secured, the most important differential activity was observed over contralateral centro-parietal regions. These patterns were larger while subjects were grasping and started to lift the object and dropped before the object was placed back.

ACCEPTED MANUSCRIPT

M AN U

SC

RI PT

Figure 2b shows the grand averaged signals centered around the grasping onset for contra- and ipsi-lateral centro parietal channels (CP3 and CP4, see Supplementary Figure S4 for other channels). The temporal evolution differed between power and precision grasps both in phase and amplitudes. These differences were significant (Bonferroni-corrected paired t-tests, p<0.05, Cohen's d corrected for paired t-tests d=1.40) for temporal periods close to the grasping onset over contra-lateral centro-parietal regions. Source localization analysis on the temporal domain also revealed significant differences at bilateral centro-parietal locations (maximum at X=-10, Y=-50, Z=70, best matches contralateral Brodmann areas 5 and 7, p=0.028, see Figure 2c), in the window of 50 milliseconds prior to the grasping onset. No significant differences were found for any other source or temporal locations. While these differences were found prior to the grasping onset, they did not trace back to the movement initiation. Indeed, no significant differences were found prior to the movement onset (see Supplementary Figure S5).

EP

TE D

EMG recorded over the first dorsal interosseous (1DI) muscle is shown in Figure 2d, together with its associated r2 values. The associated EMG power steadily increased for power grasps around 200 ms before the grasping onset, with a peak at 200 ms due to spurious activity of one subject. Similarly, precision grips power also increased prior to the onset, yet it continued increasing while the object was lifted contrarily to power grips. While during most of the time these differences were significant (corrected p<0.05), the most discriminant time interval in terms of r2 was found 200 ms after the grasping onset onwards. Noticeably, the r2 of EMG after the onset was within the same range to that of the EEG starting from 100 ms before the grasping (see Figure 2a).

AC C

3.2. Spectral analysis Following the results obtained from the spatio-temporal analysis, we evaluated the event-related spectral perturbations for precision and power grips. Sustained significant differences between the grasps and baseline activity were found for both types of grasps over alpha and beta frequency bands, throughout the whole period of -500 to 0 ms (non-parametric paired permutation test corrected for multiple comparisons, p<0.01, results not shown). More importantly, significant differences (non-parametric paired permutation test corrected for multiple comparisons, p<0.05, Cohen’s d = 2.33) on spectral power were found on the upper alpha and lower beta frequency bands ([11-13.5] Hz) over ipsilateral centro-parietal channels, but not contralaterally (see Figure 3a), with the precision grasps generating a higher spectral power than the power grasps. These differences were present as early as 275 ms prior to the grasping onset. No spectral power differences were found for low frequencies, as opposed to the temporal analysis where signals

ACCEPTED MANUSCRIPT

M AN U

SC

RI PT

showed high discriminability. Contrarily, no significant differences were found neither during the movement preparation (window [-500, 0] ms before movement onset), nor during the early reaching phase (window [0, 500] ms before movement onset) surrounding the movement onset (non-parametric permutation tests, p>0.05 for all conditions and electrodes), see Supplementary Figure S6, indicating that the grasping process was discriminable only during the period [-500, 0] ms before grasping onset. These results were confirmed by the sLoreta inverse solution, where we studied the significant differences between precision and power grasps at the intracortical level for each frequency band (see Figure 3b). No significant voxel activations were found for delta, theta, lower-alpha, beta nor gamma frequency bands (non-parametric paired permutation test corrected for multiple comparisons, p>0.1). Statistical tests revealed a significant voxel activation in ipsilateral parietal cortex (p = 0.048, corrected p-value) in upper alpha frequency band (coordinates [X = 40, Y = 70, Z = 40], Brodmann area 39), with a higher activation on the precision with respect to the power grasps.

AC C

EP

TE D

3.3. Single-trial classification We evaluated the discriminability of power vs precision grips using a linear classifier and features in the temporal domain. A grid search analysis revealed that most of the differences between the two grasping types came from two different groups of band-pass filtered signals: [1-6] Hz, and [8-15] Hz (see Supplementary Figure S7). For the sake of space, we report here the results obtained with the filter within [1-6] Hz, as they presented the highest performances. Features were extracted from the filtered EEG from contra and ipsilateral motor and parietal regions from a time window of 500 ms prior to the grasping onset (see methods for details). Figure 4a, left shows the ten-fold crossvalidation hit rates obtained for each subject and grasping type. The decoding performance (measured by the hit rate) was of 69.82±2.52% and 69.34±2.27% (mean±SEM) for power and precision grasps, respectively. Importantly, performance of all subjects was above chance level. In order to assess the spatial contribution of this discriminability, we performed the same classification approach for each EEG channel separately (see Figure 4a, right). Similarly to the results obtained with the temporal analysis, highest hit rates were obtained for contralateral centro-parietal electrodes. Contralateral frontal electrodes and ipsilateral centro-parietal channels also carried discriminable information, although their contribution was more variable across subjects (see Supplementary Figure S8). Importantly, these spatial patterns closely reflected those obtained at the sensor level and at the source space, see Figure 2a,c. The hit rate of an EMG-based classifier in an equivalent window was better than the EEG hit rates, although not significantly different: 84.60±5.41% and

ACCEPTED MANUSCRIPT

M AN U

SC

RI PT

70.44±4.06% (Bonferroni corrected paired t-tests between EMG and EEG hit rates, p = 0.09 and p = 1 for power and precision grips, respectively). While the type of grasping was distinguishable during the hand pre-shaping, it was not so during movement preparation ([-500, 0] ms before movement onset), during early reaching ([0, 500] ms before movement onset), nor during rest ([-1000, 500] ms before movement onset), in line with the findings of the spectral analysis (see Supplementary Figure S9). We further evaluated the discriminability of EEG signals between both grasping types adding information after the grasping onset (i.e., including the lifting phase) (Figure 4b). Regarding EEG, the performance increased significantly (76.60±2.04% and 79.66±2.44% hit rate for power and precision grips, p < 0.05). The spatial classification pattern (Figure 4b, right) showed similar bilateral motor and parietal activations, and frontal activations to a lesser extent. Expectedly, EMG classification improved as well, reaching 85.10±4.57% and 90.30±2.25% for power and precision grasps (significantly better than EEG for precision grasps p = 0.01, but not for power p = 0.36).

AC C

EP

TE D

3.4. Functional connectivity We computed the evolution of functional connectivity of precision vs power grips throughout the reaching and grasping for pre-motor, motor and parietal regions. We selected pre-motor, motor, and parietal regions (represented here as locations BA6, BA4 and BA7, see methods for details). No significant differences were found on delta, theta, beta nor gamma frequency bands (nonparametric permutation tests, p>0.05, corrected p-value, Figure not shown). Figure 5 shows the significantly differently activated networks for the upper alpha frequency band. Contrarily, significant differences were found in the alpha frequency bands, with most of the differences concentrated on upper alpha (p<0.05, Cohen’s d = [0.73-1.46] for the weakest and strongest significant differences). Differences between precision and power grasps during the whole period concentrated on inter-hemispheric parietal interactions; and became more evident as the grasping onset approached. As early as 100 ms prior to the grasping, we found a stronger fronto-parietal interaction for precision grasps, as well as a highly significant (p < 0.01) interaction between contra-lateral motor regions and the ipsilateral cortex.

ACCEPTED MANUSCRIPT

4. Discussion

RI PT

In this work, we have shown the existence of EEG discriminable patterns in self-paced movements for two of the most common hand grasping types, power and precision grips. Statistically significant patterns were found at the temporal, spectral and source levels, as well as at the network level via functional connectivity. Importantly, we have shown how these two graspings can be detected on a single-trial basis, with performances of around 70%.

AC C

EP

TE D

M AN U

SC

4.1. Parietal activations Results obtained by the inverse solution were in line to previous works studying precision vs power grip activations with fMRI. Ehrsson et al. [11] showed how precision grips led to stronger activations over the ipsilateral intraparietal sulcus (IPS). We found similar patterned differences, where upperalpha frequencies showed an increase in activation over the ipsilateral Brodmann Area 39 (angular gyrus), which is bounded dorsally by the IPS (and in the vicinity of BAs 5, 7 and 40, other regions found to be activated in the current study). Many studies have suggested that the anterior intraparietal area (AIP) in monkeys and its human homologue (IPS), play a key role in graspspecific tasks [1, 9, 3], mainly by providing visuospatial information about the intrinsic properties of the object [12, 35, 10]. Interestingly, different studies have consistently found that the degree of activation of this region (from no activation, contralateral, to bilateral activation) correlates with the degree of fine control needed during the reaching- and-grasping task. While reaching and pointing activated the IPS, grasping tasks activated it more strongly [14, 7]; peripheral-vision more than central-vision reaching [36], and precision grasps showed a more bilateral and stronger activation than power grasps [11, 12]. Our results showing an increase in upper-alpha power for precision grasps, suggest an inhibition of the ipsilateral parietal regions [37]. Such upper-alpha power increase has been found when subjects inhibit movements [38] and related to lower cortical excitability [39]. In particular, while lower alpha seems to be a non-specific general movement correlate, upper alpha was shown to be more specific, dissociating for example, hand from foot movements [40]. In a recent study, when subjects were asked to imagine a grasp depending on the orientation of a cylinder, high demand trials led to a selective alpha power increase in the ipsilateral cortex [41]. A follow-up study showed that when applying transcranial alternating current stimulation at 10 Hz over the ipsilateral sensorimotor cortices led to faster movement selection [42]. These studies suggest that the alpha power increase we found in the ipsilateral

ACCEPTED MANUSCRIPT

hemisphere for precision grasps could be related to the higher demands associated with these types of grasps and this, independently from movement parameters.

TE D

M AN U

SC

RI PT

4.2. Temporal evolution While recent works have pointed out the necessity of more finely grained analyses on the spatio-temporal evolution of reaching and grasping correlates [14, 15], there is still a lack of such knowledge due to the low temporal resolution or limited brain coverage of standard acquisition methods. We showed how the temporal evolution of the broadband EEG resembles temporal patterns previously observed in ECoG recordings with humans [20] and in LFPs with non-human primates [4]. A novel finding of the present work lies on the spatio-temporal evolution of such activations, where the difference between precision and power grips presented a complex pattern: During the early preshaping phase (at roughly 400 ms before the grasping onset), precision and power grasps differed in the activation over contra-lateral motor and parietal areas. As we approach the moment of the grasping onset, 100 ms before the grasping onset, we found a difference over the medial pre-frontal cortex and bilateral motor and parietal regions. Once the grasping was secured, activation dropped rapidly. These results further proved a distinctive temporal activation pattern for each grasping type, as previously reported in the aforementioned works. Whereas several fMRI studies have demonstrated that such areas were more activated during precision than power grips [11, 8, 21], we show for the first time their activation and evolution throughout the reaching-and-grasping task.

AC C

EP

4.3. Cortical grasping network EEG recordings allowed us to identify the brain networks with stronger activation during precision versus power grips. Functional connectivity between frontal, motor and parietal cortices revealed extensive inter-hemispheric differences in functional connectivity throughout the precision-grasping preshaping of the hand. In line with the spectral and inverse analyses, these differences were mostly present in the upper-alpha frequency band, as suggested previously [43]. Furthermore, we found differences in connectivity within the fronto-parietal network, suggesting a top-down process. The existence of such network is widely accepted (e.g. [44, 10]). Based on fMRI results, Grol et al. [8] proposed a model with no frontal to parietal interactions during power vs precision grips, but rather the grasping network was structured as visuo-parieto-premotor-motor unidirectionally. On the other hand, more recent works have proposed the theory that online control and correction

ACCEPTED MANUSCRIPT

EP

TE D

M AN U

SC

RI PT

during visually-guided grasping may activate opposite directionalities. Indeed, current opinions suggest that IPS and the ventral premotor cortex (PMv) present recurrent feedback loops, with the IPS performing iterative comparisons between an efference copy of the motor command and the incoming sensory information to ensure that the current reach and/or grasp is successfully executed [12, 15, 45, 46, 26]. Our results show a stronger frontoparietal activation, suggestive of a top-down influence of pre-frontal regions on parietal and motor cortices, in line with the views of alpha (as well as beta) as representing top-down control [37, 47, 27]. Moreover, these differences in connectivity started roughly after the time with maximum grip aperture (-300 ms) [1]. Altogether, this suggests that late stages of the precision grasps may require more fine motor control than the power grasps, thus the stronger interaction. Finally, stronger activations of bilateral IPS as those found here at the network level could be associated to the presence of such iterative corticocortical networks, as suggested before in this discussion. Noteworthy, this network is often interhemispheric. Other works have already shown that precision grips activate bilaterally premotor, motor and parietal regions [21]. However, they have not studied the interaction among different brain regions. In our work, the ipsilateral motor cortex seemed to be critical during the period just before the grasping is secured. Although the exact contribution of ipsilateral motor cortex is still disputed, it is thought that it may contribute to finely tune the motor command originating from the contralateral motor cortex in tasks requiring so via transcallosal influences [48], with higher muscle selectivity implying stronger ipsilateral activations [49]. Following this theory, and despite results obtained in this work have to be considered with caution, a possible interpretation is that pre-motor and parietal cortices may inform the ipsilateral motor cortex about fine aspects of the motor task that needs to be executed.

AC C

4.4. Artifact contamination and physiological confounds Great care was taken to ensure that no muscular or electrooculographic artifacts corrupted our recordings. Regarding ocular artifacts, subjects were instructed to fixate their gaze to a cross in front of them and not to sac- cade towards the grasped object. Further removal of possibly remaining ocular artifacts was performed using a linear regression algorithm which has demonstrated its high performance in removing such activity [28] (see Supplementary Figure S1). As for muscular activity, subjects did not make large head movements during the recording. Moreover, the differences found over parietal areas are unlikely to be related to muscular artifacts, as these artifacts are usually well confined over temporal regions. Finally, the contamination due

ACCEPTED MANUSCRIPT

AC C

EP

TE D

M AN U

SC

RI PT

to these artifacts tends to be present on higher frequencies [50]. Other artifacts were also controlled. First, for cable movements artifacts, cables were fixed to the chair with adhesive tape in an attempt to minimize such movements. Finally, gel-electrode coupling variations were unlikely, as the task followed standard procedures within the EEG field mainly ensuring a good impedance prior the experiment and limiting the EEG recording as such to a maximum of one hour. Notwithstanding, and in order to confirm the absence of such artifacts, we performed an a-posteriori analysis with an additional artifact removal step. Using independent component analysis (ICA) [51], we removed those components that were considered as artifactual based on prior physiological information (see Supplementary Materials, and Supplementary Figures S10 and S11 for details). After artifact removal with ICA, we recomputed the classification analysis within the window [-500, 0] ms, and found no significant differences in decoding performance before and after artifact removal (power grasps hit rate: 69.53±2.51%, t9 = -0.73, p=0.48 paired t-test versus original hit rates; precision grasps hit rate: 70.44±2.29%, t9 = -0.61, p=0.56). Thus, we concluded that the decoding performance was not affected by artifactual components. These considerations, together with the anatomical plausibility of the activations found, led us to the conviction that EEG can be used as a tool for studying complex reach-and-grasp tasks. An intrinsic limitation of our experimental protocol was that, by design, power and precision grasps may require a different wrist pronation/supination, as well as different forces applied during different graspings. Regarding the force, Pisthol and colleagues did not find this parameter to be driving the decoding performance during precision vs power grasps using ECoG recordings [20], although Jochumsen et al. reported discriminable force differences of lower-limb movements using EEG [52]. As for the kinematics, Ofner and colleagues have recently shown how low-level kinematic parameters such as upper-limb pronation/supination, or wrist rotations can be classified using lowfrequency EEG (below 3 Hz) [53]. Similarly, Jochumsen et al. showed that speed differences can also be discriminated, yet at the lower-limb level [52]. In our work, speed was not a confounding factor, as there was no correlation between differences in speed and decoding performances (r=-0.26, p=0.47). As for the rest of the parameters, although we cannot dissociate whether our grasping decoding comes from such correlates or not, this was not the objective of our study. Instead, we argue that in complex movements such as grasping, wrist pronation is only one of many parameters that can differ and it would be very difficult to control for every parameter: grasp force, limb velocity, hand opening or muscle synergies among others. In the current work,

ACCEPTED MANUSCRIPT

M AN U

SC

RI PT

we thus considered grasping as a high-level combination of these lower level parameters under an ecological non-controlled condition. As another limitation, our experimental design made that the precision grasps were always located slightly to the right of power grasps. As such, we cannot fully discard the possibility that a directional effect may have confounded our results. However, we believe this is unlikely, and also that our experimental design still accounted for this effect for two reasons: First, the distance between the power and precision positions was very small (approximately 2 cm), while decoding of directions using EEG has been shown to be above significance with distances one order of magnitude larger (e.g. see [54]). Second, although it is true that the one grasp was located more to the right than the other, the object repositioning covered the whole field of view of the subjects, including centered positions as well as each of the two hemispheres, and at different distances. When pooled together, we believe that this confound is removed in terms of mapping, inter-hemispheric connectivity, and especially decoding results as those found here.

AC C

EP

TE D

4.5. Implications on brain-machine interfaces While the use of EEG for the decoding of motor correlates such as reaching intention [17, 18, 55] or directionality [56, 54] has been studied for a long time, EEG-based decoding of other correlates such as grasping has received less attention in the community. Although remarkable results have been obtained with fMRI [21], as well as with invasive recordings using LFP [4], ECoG [20], and neural spiking rates [3], the works of Agashe et al. [23], Schwarz et al. [24], and Jochumsen et al. [25] are among the few on EEG. The present work demonstrates how two hand grasping types can be decoded offline from EEG with a decoding performance close to 70%, using information prior to the grasping onset, or 80% using also information after the onset. The use of single-trial classification further validated the existence of differential neural correlates of grasping types. An important remark on the findings of this work is that, although statistical differences can be found at the group level (e.g. 8-15 Hz increase in power as proved by the ERSP analysis), this does not necessarily imply that they will be the ones leading to a higher decoding performance. Indeed, our work showed that highest performances were obtained in the 1-6 Hz filtered EEG, due to the spatio- temporal combination of different EEG channels and time points. It is worth noting that our goal is not to show that EEG decoding is better than EMG, nor that it can decode grasping types earlier in time. Rather, our goal was to reveal the existence of correlates, and show that they can be decoded in single trial with brain signals. In fact, for several motor conditions

ACCEPTED MANUSCRIPT

AC C

EP

TE D

M AN U

SC

RI PT

such as spinal-cord injury or amputees, the timing of decoding of EEG is not the key issue, but rather to actually be able to decode the signals where EMG fails. Nonetheless, an important future study should address whether EEG can provide additional information to EMG decoders, and how they can be combined to improve performance for neuroprosthesis aimed at users with some level of motor residual capabilities [57]. Similar findings on EEG-based classification have been made by previous approaches. Using low-frequency EEG (below 1 Hz), Agashe et al. reported closed-loop decoding of joint angles and movement synergies in grasping tasks, with successful results in both healthy subjects and transradial amputees [23]. More recently, Schwarz et al further reported how different types of grasping could be decoded in healthy subjects from low-frequency EEG components (below 3 Hz) [24]. In our work, we corroborated those results and performed an additional battery of tests to find the most discriminant frequency bands. Here, we showed that a broader bandpass filter can be beneficial in terms of decoding performance. Noticeably, we did not obtain performances higher than chance level during the pre-movement preparation phase, contrarily to previous findings [23, 24, 25]. A main difference in our work is, however, that subjects were not cued neither when to grasp nor which grasping to perform, which thus could explain the seemingly contradicting results. Indeed, a large body of works has studied the differences between self-paced and cued movements and have shown how they involve functionally separated cortico-basal ganglia networks [58, 59]. Further experiments are thus needed to evaluate these differences, which are key for the successful development of a neural prosthesis. Due to the offline characterization nature of our work, and in order to maintain homogeneity with the rest of analysis, we opted for the use of a zerophase non-causal bandpass filter for the signals, as they provide a better quantification in terms of accurate timing estimation [60]. However, a real-time closed-loop system cannot make use of such filters, and one has to opt for the use of causal bandpass filters. For this reason, we performed an additional decoding analysis using an equivalent, causal filter (within the time window [500, 0] ms). The performances obtained were of 67.77±2.21% and 62.64±3.13% for power and precision grasps respectively. These results were not significantly worse than the decoding with a non-causal filter for power grasps (Bonferroni corrected paired t-test, t9 = -2.11, p = 0.13), but they were for precision grasps (t9 =-2.72, p = 0.047). Importantly, however, this drop in performance was limited (4.40% on average), and performance was still above chance level, allowing for its use on closed-loop. Future works will need to be performed to evaluate whether these performances can be improved with the

ACCEPTED MANUSCRIPT

AC C

EP

TE D

M AN U

SC

RI PT

use of closed-loop feedback, as it is customary when using these systems. Additional experiments will be needed to study the patterns generated by imagined grasping movements and during online experiments. In this regard, Klaes et al. have shown the possibility of decoding imagined hand gestures from the posterior parietal cortex from spiking rates [3]. Whether imagined graspings can be decoded with EEG, however, is yet to be elucidated, although recent works suggest that novel types of feedback using sensory neuromuscular electrical stimulation could help in the decoding of such correlates [61]. While targeting the development of a neuroprosthesis executing different grasping types is attractive, these correlates will be insufficient to develop an upper-limb prosthesis for reaching and grasping. The decoder presented here should form part of a cascaded classifier with an earlier detection system able to estimate grasping onsets from neural correlates [62]. Importantly, recent works have shown that such signals are present in ECoG [6], and we have corroborated those findings using EEG in one of our previous works [19]. We believe that a neuroprosthetic framework where an intelligent controller integrates high-level motor correlates of reaching and grasping may be a possible alternative and have advantages with respect to classical kinematicbased approaches, such as reliability and robustness due to the inclusion of higher-level shared-control techniques [63, 64, 65], for instance integrated within portable exoskeletons for daily living activities [66].

ACCEPTED MANUSCRIPT

5. Author Contributions

RI PT

Conceptualization, I.I., R.C., R.L. and J.M.; Methodology, I.I., H.Z., T.C., and M.P.; Investigation, I.I.; Writing Original Draft, I.I.; Writing Review and Editing, All authors; Funding Acquisition, R.C. J.M.; Resources, R.C., R.L. and J.M.; Supervision, R.C., R.L. and J.M. 6. Acknowledgments

SC

This work has been supported by the Swiss National Centres of Competence in Research (NCCR) Robotics. I.I. also acknowledges support from the EPFL Fellows fellowship programme co-funded by Marie Curie, FP7 Grant agreement no. 291771.

M AN U

7. References

[1] U. Castiello, The neuroscience of grasping, Nature Reviews Neuroscience 6 (2005) 726–736. [2] S. Schaffelhofer, H. Scherberger, Object vision to hand action in macaque parietal, premotor, and motor cortices, eLife 5 (2016) e15278.

TE D

[3] C. Klaes, S. Kellis, T. Aflalo, B. Lee, K. Pejsa, K. Shanfield, S. Hayes-Jackson, M. Aisen, C. Heck, C. Liu, R. Andersen, Hand shape representations in the human posterior parietal cortex, Journal of Neuroscience 35 (2015) 15466–15476.

EP

[4] M. Mollazadeh, V. Aggarwal, G. Singhal, A. Law, A. Davidson, M. Schieber, N. Thakor, Spectral modulation of LFP activity in M1 during dexterous finger movements, in: 30th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, 2008, pp. 5314–5317.

AC C

[5] S. Acharya, M. Fifer, H. Benz, N. Crone, N. Thakor, Electrocorticographic amplitude predicts finger positions during slow grasping motions of the hand, Journal of Neural Engineering 7 (2010) 046002.

[6] T. Pistohl, T. Schmidt, S. Benedikt, T. Ball, A. Schulze-Bonhage, Aertsen, C. Mehring, Grasp detection from human ECoG during natural reach-to-grasp movements, PloS One 8 (2013) e54658.

[7] J. Culham, S. Danckert, J. De Souza, J. Gati, R. Menon, M. Goodale, Visually guided grasping produces fMRI activation in dorsal but not ventral stream brain areas, Experimental Brain Research 153 (2003) 180–189.

ACCEPTED MANUSCRIPT

[8] M. Grol, J. Majdandzic, K. Stephan, L. Verhagen, H. Dijkerman, H. Bekkering, F. Verstraten, I. Toni, Parieto-frontal connectivity during visually guided grasping, Journal of Neuroscience 27 (2007) 11877– 11887.

RI PT

[9] C. Begliomini, M. Wall, A. Smith, U. Castiello, Differential cortical activity for precision and whole-hand visually guided grasping in humans, European Journal of Neuroscience 25 (2007) 1245–1252. [10] J. Gallivan, D. McLean, K. Valyear, J. Culham, Decoding the neural mechanisms of human tool use, eLife 2 (2013) e00425.

SC

[11] H. Ehrsson, A. Fagergren, T. Jonsson, G. Westling, R. Johansson, H. Forssberg, Cortical activity in precision versus power-grip tasks: An fMRI study, Journal of Neurophysiology 83 (2000) 528–536.

M AN U

[12] E. Olivier, M. Davare, M. Andres, L. Fadiga, Precision grasping in humans: From motor control to cognition, Current Opinion in Neurobiology 17 (2007) 644–648. [13] M. Davare, R. Lemon, E. Olivier, Selective modulation of interactions between ventral premotor cortex and primary motor cortex during precision grasping in humans, The Journal of Physiology 586 (2008) 2735– 2742.

TE D

[14] F. Filimon, Human cortical control of hand movements: Parietofrontal networks for reaching, grasping, and pointing, The Neuroscientist 16 (2010) 388–407. [15] U. Castiello, C. Begliomini, The cortical control of visually guided grasping, The Neuroscientist 14 (2008) 157–170.

EP

[16] D. Koester, T. Schack, J. Westerholz, Neurophysiology of grasping actions: Evidence from ERPs, Frontiers in Psychology 7 (2016).

AC C

[17] I. Niazi, N. Jiang, O. Tiberghien, J. Nielsen, K. Dremstrup, D. Farina, Detection of movement intention from single-trial movement-related cortical potentials, Journal of Neural Engineering 8 (2011) 066009. [18] E. Lew, R. Chavarriaga, S. Silvoni, JdR. Millán, Detection of self-paced reaching movement intention from EEG signals, Frontiers in Neuroengineering 5 (2012). [19] L. Randazzo, I. Iturrate, R. Chavarriaga, R. Leeb, JdR. Millán, Detecting intention to grasp during reaching movements from EEG, in: 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society), 2015, pp. 1115–1118.

ACCEPTED MANUSCRIPT

[20] T. Pistohl, A. Schulze-Bonhage, A. Aertsen, C. Mehring, T. Ball, Decoding natural grasp types from human ECoG, Neuroimage 59 (2012) 248–260.

RI PT

[21] D. Bono, M. Grazia, C. Begliomini, U. Castiello, M. Zorzi, Probing the reaching–grasping network in humans through multivoxel pattern decoding, Brain and Behavior 5 (2015). [22] H. A. Agashe, A. Y. Paek, Y. Zhang, J. L. Contreras-Vidal, Global cortical activity predicts shape of hand during grasping, Frontiers in neuroscience 9 (2015).

SC

[23] H. Agashe, A. Paek, J. Contreras-Vidal, Multisession, noninvasive closedloop neuroprosthetic control of grasping by upper limb amputees, Progress in Brain Research 228 (2016) 107–128.

M AN U

[24] A. Schwarz, P. Ofner, J. Pereira, A. I. Sburlea, G. R. Müller-Putz, Decoding natural reach-and-grasp actions from human EEG, Journal of neural engineering (2017). [25] M. Jochumsen, I. K. Niazi, K. Dremstrup, E. N. Kamavuako, Detecting and classifying three different hand movement types through electroencephalography recordings for neurorehabilitation, Medical & biological engineering & computing 54 (2016) 1491–1501.

TE D

[26] M. Davare, A. Kraskov, J. Rothwell, R. Lemon, Interactions between areas of the cortical grasping network, Current Opinion in Neurobiology 21 (2011) 565–570. [27] P. Fries, Rhythms for cognition: Communication through coherence, Neuron 88 (2015) 220–235.

EP

[28] A. Schlögl, C. Keinrath, D. Zimmermann, R. Scherer, R. Leeb, G. Pfurtscheller, A fully automated correction method of EOG artifacts in EEG recordings, Clinical Neurophysiology 118 (2007) 98–104.

AC C

[29] R. Pascual-Marqui, Standardized low-resolution brain electromagnetic tomography (sLORETA): Technical details, Methods Find Exp Clin Pharmacol 24 (2002) 5–12. [30] E. Maris, R. Oostenveld, Nonparametric statistical testing of EEG and MEG data, Journal of Neuroscience Methods 164 (2007) 177–190. [31] A. Ramos-Murguialday, N. Birbaumer, Brain oscillatory signatures of motor tasks, Journal of Neurophysiology 113 (2015) 3663–3682.

ACCEPTED MANUSCRIPT

[32] I. Iturrate, L. Montesano, R. Chavarriaga, JdR. Millán, J. Minguez, Spatiotemporal filtering for EEG error related potentials., in: 5th Int BrainComputer Interface Conference, volume 1, 2011, pp. 12–15.

RI PT

[33] B. Blankertz, S. Lemm, M. Treder, S. Haufe, K. Müller, Single-trial analysis and classification of ERP components: A tutorial, NeuroImage 56 (2011) 814–825.

SC

[34] R. D. Pascual-Marqui, D. Lehmann, M. Koukkou, K. Kochi, P. Anderer, B. Saletu, H. Tanaka, K. Hirata, E. R. John, L. Prichep, et al., Assessing interactions in the brain with exact low-resolution electromagnetic tomography, Philosophical Transactions of the Royal Society of London A: Mathematical, Physical and Engineering Sciences 369 (2011) 3768–3784.

M AN U

[35] M. Davare, J. Rothwell, R. Lemon, Causal connectivity between the human anterior intraparietal area and premotor cortex during grasp, Current Biology 20 (2010) 176–181. [36] J. Prado, S. Clavagnier, H. Otzenberger, C. Scheiber, H. Kennedy, M. Perenin, Two cortical systems for reaching in central and peripheral vision, Neuron 48 (2005) 849–858. [37] W. Klimesch, P. Sauseng, S. Hanslmayr, EEG alpha oscillations: The inhibition-timing hypothesis, Brain Research Reviews 53 (2007) 63–88.

TE D

[38] P. Sauseng, C. Gerloff, F. C. Hummel, Two brakes are better than one: The neural bases of inhibitory control of motor memory traces, NeuroImage 65 (2013) 52–58.

EP

[39] P. Sauseng, W. Klimesch, C. Gerloff, F. C. Hummel, Spontaneous locally restricted EEG alpha activity determines cortical excitability in the motor cortex, Neuropsychologia 47 (2009) 284–288.

AC C

[40] G. Pfurtscheller, C. Neuper, G. Krausz, Functional dissociation of lower and upper frequency mu rhythms in relation to voluntary limb movement, Clinical Neurophysiology 111 (2000) 1873–1879. [41] L. Brinkman, A. Stolk, H. C. Dijkerman, F. P. de Lange, I. Toni, Distinct roles for alpha- and beta-band oscillations during construction of goal-directed action plans, Journal of Neuroscience 34 (2014) 14783–14792. [42] L. Brinkman, A. Stolk, T. R. Marshall, S. Esterer, P. Sharp, H. C. Dijkerman, F. P. de Lange, I. Toni, Independent causal contributions of alpha- and beta-band oscillations during movement selection, Journal of Neuroscience 36 (2016) 8726–8733.

ACCEPTED MANUSCRIPT

[43] R. Spinks, A. Kraskov, T. Brochier, M. Umilta, R. Lemon, Selectivity for grasp in local field potential and single neuron activity recorded simultaneously from M1 and F5 in the awake macaque monkey, Journal of Neuroscience 28 (2008) 10961–10971.

RI PT

[44] G. Koch, M. Cercignani, C. Pecchioli, V. Versace, M. Oliveri, C. Caltagirone, J. Rothwell, M. Bozzali, In vivo definition of parieto-motor connections involved in planning of grasping movements, Neuroimage 51 (2010) 300– 312.

SC

[45] G. Mulliken, S. Musallam, R. Andersen, Forward estimation of movement state in posterior parietal cortex, Proceedings of the National Academy of Sciences 105 (2008) 8170–8177.

M AN U

[46] R. Andersen, H. Cui, Intention, action planning, and decision making in parietal-frontal circuits, Neuron 63 (2009) 568–583. [47] S. Sadaghiani, A. Kleinschmidt, Brain networks and α-oscillations: Structural and functional foundations of cognitive control, Trends in cognitive sciences 20 (2016) 805–817. [48] M. Davare, J. Duque, Y. Vandermeeren, J. Thonnard, E. Olivier, Role of the ipsilateral primary motor cortex in controlling the timing of hand muscle recruitment, Cerebral Cortex 17 (2007) 353–362.

TE D

[49] T. Verstynen, J. Diedrichsen, N. Albert, P. Aparicio, R. Ivry, Ipsilateral motor cortex activity during unimanual hand movements relates to task complexity, Journal of Neurophysiology 93 (2005) 1209–1222.

EP

[50] I. Goncharova, D. McFarland, T. Vaughan, J. Wolpaw, EMG contamination of EEG: spectral and topographical characteristics, Clinical Neurophysiology 114 (2003) 1580–1593.

AC C

[51] A. Delorme, S. Makeig, Eeglab: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis, Journal of neuroscience methods 134 (2004) 9–21. [52] M. Jochumsen, I. K. Niazi, N. Mrachacz-Kersting, D. Farina, K. Dremstrup, Detection and classification of movement-related cortical potentials associated with task force and speed, Journal of neural engineering 10 (2013) 056015. [53] P. Ofner, A. Schwarz, J. Pereira, G. R. Müller-Putz, Upper limb movements can be decoded from the time-domain of low-frequency EEG, PloS one 12 (2017) e0182578.

ACCEPTED MANUSCRIPT

[54] E. Lew, R. Chavarriaga, S. Silvoni, JdR. Millán, Single trial prediction of selfpaced reaching directions from EEG signals, Frontiers in Neuroscience 8 (2014) 222.

RI PT

[55] E. López-Larraz, L. Montesano, A. Gil-Agudo, J. Minguez, Continuous decoding of movement intention of upper limb self-initiated analytic movements from pre-movement EEG correlates, Journal of Neuroengineering and Rehabilitation 11 (2014) 1–15.

SC

[56] S. Waldert, H. Preissl, E. Demandt, C. Braun, N. Birbaumer, A. Aertsen, C. Mehring, Hand movement direction decoded from MEG and EEG, Journal of Neuroscience 28 (2008) 1000–1008.

M AN U

[57] D. Novak, X. Omlin, R. Leins-Hess, R. Riener, Predicting targets of human reaching motions using different sensing technologies, IEEE Transactions on Biomedical Engineering 60 (2013) 2645–2654. [58] A. M. Graybiel, Habits, rituals, and the evaluative brain, Annual Review of Neuroscience. 31 (2008) 359–387.

TE D

[59] O. Bichsel, R. Gassert, L. Stieglitz, M. Uhl, H. Baumann-Vogel, D. Waldvogel, C. R. Baumann, L. L. Imbach, Functionally separated networks for self-paced and externally-cued motor execution in Parkinson’s Disease: Evidence from deep brain recordings in humans, Neuroimage 177 (2018) 20-29. [60] D. Yael, J. J. Vecht, I. Bar-Gad, Filter based phase shifts distort neuronal timing information, eNeuro 5 (2) (2018) ENEURO.0261-17.2018.

EP

[61] T. Corbet, I. Iturrate, M. Pereira, S. Perdikis, JdR. Millán, Sensory threshold neuromuscular electrical stimulation fosters motor imagery performance, Neuroimage 176 (2018) 268–276.

AC C

[62] J. Omedes, A. Schwarz, L. Montesano, G. Müller-Putz, Hierarchical decoding of grasping commands from EEG, in: Engineering in Medicine and Biology Society (EMBC), 2017 39th Annual International Conference of the IEEE, IEEE, 2017, pp. 2085–2088. [63] R. Andersen, E. Hwang, G. Mulliken, Cognitive neural prosthetics, Annual Review of Psychology 61 (2010) 169. [64] T. Carlson, JdR. Millán, Brain-controlled wheelchairs: A robotic architecture, IEEE Robotics and Automation Magazine 20 (2013) 65–73.

ACCEPTED MANUSCRIPT

[65] I. Iturrate, R. Chavarriaga, L. Montesano, J. Minguez, JdR. Millán, Teaching brain-machine interfaces as an alternative paradigm to neuroprosthetics control, Scientific Reports 5 (2015), 13893.

AC C

EP

TE D

M AN U

SC

RI PT

[66] L. Randazzo, I. Iturrate, S. Perdikis, JdR. Millán, mano: A wearable hand exoskeleton for activities of daily living and neurorehabilitation, IEEE Robotics and Automation Letters 3 (2018) 500–507.

ACCEPTED MANUSCRIPT

RI PT

Figure 1: Experimental protocol. (a) Schematic illustration of the setup. For each trial, subjects started with their dominant hand on the rest trigger, where they had to stay at least for 2 seconds. Then, they reached and grasped the object (using a power grasp or precision pinch, in two different positions signaled by two triggers), lifted the object and returned with their hand to the rest trigger. (b) Timeline of a single trial. 0 ms marks the moment considered as grasping onset. In this work, we mostly evaluated the grasping pre-shaping phase, see shadowed area; although we also present results after the grasping is secured (i.e. lifting phase).

M AN U

SC

Figure 2: Temporal analysis. (a) Whole scalp temporal evolution of differences between 2 precision and power grips computed as a discriminability score (r ), where time 0 ms indicates the onset of grasping. (b) Grand average (mean ± standard error of the mean, solid lines ± shadowed regions) of the two most representative EEG channels, CP3 and CP4. (c) Source localization at the temporal domain. Red colors denote an increase in activity for precision grips compared to power grips. For the sake of simplicity, only significant windows are shown. (d) Grand average of the EMG recorded over the 1DI. Similarly to panel (a), the discriminability index is shown below the EMG average. Significant differences are marked above the average. Significant differences (p<0.05 corrected for multiple comparisons) for all analyses are marked with green regions.

TE D

Figure 3: Spectral analysis (a) Event-related spectral perturbations for the two most representative channels (CP3 and CP4), where horizontal and vertical axes represent the time and frequency, respectively. Blue and yellow colors represent weaker and stronger power for precision grips when compared to power grips, respectively. Dashed and solid lines represent p-values below 0.1 and 0.05, computed from a non-parametric permutation test. (b) Spectral source localization results. Significant differences (corrected p<0.05) were found between precision and power grips only in the upper alpha ([10-12] Hz) frequency band. Red colors denote an increase in activity for precision grips compared to power grips.

EP

Figure 4: Single-trial classification results. Classification results of detecting precision vs power grips, using features (a) prior to grasping onset (window [-500, 0] ms), and (b) prior and after the grasping onset (window [-500, 500] ms). Results are shown for each subject separately, on average, as well as the performance obtained with the EMG channel. The right panels show a topographic interpolation of performance obtained for each separate channel and averaged over all subjects.

AC C

Figure 5: Functional connectivity at the source space. Lagged phase synchronization at the source space. Only significant interactions are shown (thin line: p<0.05; thick line: p<0.01). Larger (smaller) activations for the precision grip are represented in red (blue). Significant interactions were found only in the upper alpha frequency band. Each column represents a time bin, from -400 ms with steps of 100 ms and a window width of 500 ms. The precision grip almost always exhibited larger connectivity than power grips.

a

ACCEPTED MANUSCRIPT

SC

RI PT

Power grasp

b

Rest trigger unpressed

REST >2000 ms

AC C

EP

TE D

M AN U

Precision grasp

Rest trigger

Grasping trigger pressed

REACH-TO-GRASP 0 ms

Grasping trigger unpressed

LIFT

Rest trigger pressed

RETURN Time

[0, 50] ms

b

[50, 100] ms

1

0.5 0

-0.5 -1

µ V (EMG)

40

M AN Amplitude (µV) U

[200, 250] ms

20

0 -500

EP

[-50, 0] ms

-400

CP4

-300

-200

-100

0

Time (ms)

100

200

300

Power grasp 400 500 Pinch grasp

0

0.05

-1.5

d

1DI

100

Amplitude (µV)

c

r2

AC C

0

0

-1.5 1.5

TE D

[150, 200] ms

p<0.05

CP3

-1.5

[100, 150] ms

Power grasp Power grasp Pinch grasp grasp Precision

RI PT

[-50, 0] ms

[-100,

SC

p<0.05

[-150, -100] ms

Amplitude (µV)

[-200, -150] ms

µ V (EEG channel CP3 ) Amplitude (µV)

a

1.5 1.5 MANUSCRIPT -50]ACCEPTED ms

0 r2

-400

-200

0

Time Time (ms) (ms)

200

400

ACCEPTED MANUSCRIPT p < 0.05

5

10

15 20 25 30

20 25 30

35

35

0

Time (ms)

200

40

400

-400

Upper-alpha

TE D

-200

EP

-400

M AN U

CP3

AC C

b

15

RI PT

Frequency (Hz)

Frequency (Hz)

10

0

-200

Relarive power (au)

5

40

30%

p < 0.1

SC

a

CP4

0

Time (ms)

200

400

-30%

a

100 100 100 90

90

ACCEPTED MANUSCRIPT

Power

Power Power Precision

Precision Precision

Chance level

80 80

70

80

Accuracy (%) (%) Hit rate (%) Accuracy

RI PT

60 50

40 40 50

SC

30

40

20 20

30 10

s2 s2

s3 s3

s4 s4

s5 s5

s6 s6

s7 s7

s8 s8

100 100 0 90

Power

Precision s1

s2

s3

s4

s9 s9

TE D

10 s5

s6

s7

s10 s10

s8

mean mean mean EEG mean EMG EEG

s9

EMG

s10

EP

80 80 70

Hit rate (%) (%) Accuracy

60 60

AC C

[-500, 500] ms

b

s1 s1

50

40 40 30 20 20

mean EEG mean EMG

70

50

10

00

50

Hit rate (%)

00 20

Hit rate (%)

60 60

M AN U

[-500, 0] ms

70

70

s1 s1

s2 s2

s3 s3

s4 s4

s5 s5

s6 s6

s7 s7

s8 s8

s9 s9

s10 s10

mean EEG mean EMG mean mean

EEG

EMG

ACCEPTED MANUSCRIPT

Upper alpha -300 ms

BA6

BA6

RI PT

BA6

BA4 BA6

BA4 BA6

Power > Precision

p<0.01

0 ms

-100 ms

SC

BA4

M AN U

BA4

p<0.05

BA4 BA6

BA4 BA6

BA4 BA6

BA4 BA6

AC C

BA6

BA4

TE D

BA4

-200 ms

EP

-400 ms

Precision > Power

BA7

BA7

BA7

BA7

BA7

BA7

BA7

BA7

BA7

BA7