1.1. Importance of perceptual process for goal-directed movements
Goal directed movements are organized via perceptual information that is relevant to movement situation. Even in a simple movement of reaching out a glass on a table and grasping it, the configuration and orientation of hand and fingers should be organized with respect to the size, shape, and orientation of the glass. According to the study on prehensile movements, the size of aperture shaped by an index finger and a thumb to grasp an object was organized with respect to the size of the object such that the peak aperture was observed well before the hand reaches the object and the peak value was linearly scaled to the object’s size (Jeannerod, 1981, 1984). When a mechanical perturbation was applied to an upper arm during a prehensile movement to assist or disturb the hand reaching an object, the well-coordinated reaching and grasping components was observed in terms of timing the grasping movement with respect to the moment of the hand reaching the object (Haggard & Wing, 1995). For pre-shaping the aperture and temporally organizing the reaching-grasping components, perceptual information about the object size and the time to the hand-object contact is crucial. Therefore, how perceptual process plays a role for organizing a movement and what/how perceptual information is utilized for the movement organization have been major issues in the study of motor control.
1.2. Two cortical pathways for visual information processing
According to the study on the cortical function for visual processing, there are two visual streams from the primary visual cortex to the posterior parietal cortex (the dorsal stream) and to the inferotemporal cortex (the ventral stream). Lesions to one of these visual pathways induce different types of perceptual-motor deficits. The lesions associated with the dorsal stream (e.g., the occipitoparietal region) induce the inability to shape the prehensile aperture for reaching and grasping an object properly but with no difficulty in visually discriminating one such object from another. Contrary to it, the lesions associated with the ventral stream (e.g., the ventrolateral region) leads to the reverse deficit (i.e., the inability to visually discriminate the object with the intact aperture control for grasping). Therefore, these findings have been regarded as the evidence of two visual processing pathways, one for visuomotor control via the dorsal stream, and the other for cognitive visual processing via the ventral stream (Goodale et al., 1994).
1.3. Cognitive aspect of perceptual-motor process for executing task performances
From a computational or information processing point of view(e.g., Schmidt& Lee, 1999), the cognitive process of recognizing the identity of an object to be grasped and planning how to produce a grasping movement with respect to the recognized object’s shape, size, and orientation is central for organizing a prehensile movement.Such cognitive aspect of visual information processing for achieving perceptual-motor tasks with respect to a target object has been studied by an experimental paradigm using an target object, such as, the Ebbinghaus figure or Müller-Lyer figure that induces a visual illusion about the object size.
In this experimental paradigm, the following two perceptual-motor tasks have been used: 1) reaching out toward and grasping a visual object with an index-thumb pinch grip, and 2) assessing the size of the same visual object and indicating the estimated size by the same aperture as used to grasp the object. These two tasks share a qualitatively similar perceptual-motor process in terms of producing the same aperture configuration based on the same visual information about the target figure.However, the involvement of cognitive process (i.e., recognizing the target object, estimating its size, and deciding the grasping aperture size with respect to the perceived object size) seems to be different. In the size-estimation task, the production of the aperture configuration requires explicit identification of the size of the figure and the particular aperture size needs to be associated arbitrary with respect to the particular perceived size of the figure. In this sense, executing this task is cognitive process-oriented (Ranganathan& Carlton, 2007). As for the reaching-grasping task, the study on modeling a prehensile movement with nonlinear equations of motion, which include a perceptual variable as a parameter to modulate the dynamics of the movement, demonstrated the spatial and temporal characteristics of upper limb kinematics in the prehensile motion(Schöner, 1994; Zaal, 1998). This result supports the idea in the theoretical frameworks of the ecological perspective (Lee, 1980; Turvey & Kugler, 1984; Warren, 1990) and dynamical system account (e.g., Kelso, 1995; Schöner & Kelso, 1988) for motor coordination such that organizing a prehensile movement may not necessarily involve a cognitive process, such as the object identification and the arbitral object-aperture size association. From the above perspective, the perception of a target object and an action with respect to it are mutually dependent in the grasping task, whilst those in the size-estimation task are uncoupled and mediated via the cognitive process.
The original findings in the seminal studies using the paradigm were such that visual discrimination or perception about the object’s size was susceptible to the illusory object, but the grasping aperture with respect to it was not (e.g., Aglioti et al., 1995; Haffenden & Goodale, 1998). An argument based on these findings has been such that cognitive perceptual processing and motor production process can be dissociable (Goodale & Milner, 1992) and the theoretical confrontation between the cognitive account for information processing in organizing a movement (e.g., Schmidt & Lee, 1999) and the ecological perspective for the perceptual-motor process (e.g., Lee, 1980; Turvey & Kugler, 1984; Warren, 1990) can be ascribed to these two visual streams (Tresilian, 1995). However, contradictory result has been reported such that the effect of the misperception about the object size was also observed in a prehensile movement (e.g., Franz, et al., 2000; Franz, et al., 2001). Other studies also found the susceptibility to the illusory object in a prehensile movement and suggested: the involvement of the ventral stream involved in a grasping motion with respect to a complex object (McIntosh, et al., 2004); the partial, not exclusive, dissociation between the two pathways (Ellis, et al., 1999); a multiple visuomotor process involving both pathways (Westwood, et al., 2000b); and the execution of prehensile movements by involving the ventral stream via the supplementary motor areas (Lee & van Donkelaar, 2002).
1.4. Examination of cortical activities in perceptual-motor performances
The above findings suggest that integrated function of cortical networks for executing the visuo-motor task needs to be considered for fully understanding the mechanism of the perceptual-motor process. From this view, the present study examines the cortical activation pattern during the reaching-grasping and the size-estimation performances. A particular focus for this investigation is on how cortical activities associated with the dorsal and ventral streams are involved in the perceptual-motor process for the task performances.
For this investigation, it is necessary to assess the effect of the perception of the target object size on the task execution. Therefore, the two task movements were produced with respect to a neutral object and an object inducing a size illusion (the Ebbinghaus figure). The illusion effect on the aperture configuration indicates that cognitive processing is involved in the task execution. In the case of the size-illusion effect observed in the size-matching performance but in the grasping, the observed cortical activitiesare interpreted in terms of the differences in association between cognitive processing and movement execution.If distinctive activation patterns between the two task performances are observed, it may be attributed to the difference in the perceptual-motor process associated with the involvement of cognitive processing. Conversely, if no difference in the cortical activity patterns between them, it may suggests some qualitative similarity in the cortical process between the different task executions.
On the other hand, the illusion effect on both of the tasks performances indicates that cognitive process is involved even in the reaching-grasping performance. In this case, the point of comparison in cortical activities between the two task conditions may not whether the dorsal and ventral streams are exclusively functioned, but how worked as an integrated cortical network. If difference in the pattern of cortical activities is observed, it reflects qualitative difference in the participation of cognitive process in the task movement execution.
1.5. Examination of the brain dynamics related to task execution
To investigate the cortical activity,electroencephalograph (EEG) during the task performance was analyzed in terms of frequency domain. Two different analyses, which potentially shed light on the different aspects of cortical activities, were conducted: the change of the EEG frequency power spectrum that was time-locked to the task event (Event-related spectral perturbation: Makeig, 1993; for review, Pfurtscheller et al., 1999a) and the coherence between EEGs of two electrodes (Event-related coherence: e.g., for review, Hummel & Gerloff, 2006; Schlögl & Supp, 2006; Pfurtscheller& Andrew, 1999).
Event-related spectral perturbation (ERSP) quantifies the degree to which the amplitude of a particular frequency band of ongoing EEG attenuates or enhances in response to a stimulus event, which is termed event-related desynchronization (ERD) or synchronization (ERS), respectively. ERD has been regarded as representing an activated cortical state with which the processing of sensory, motor, or cognitive information is enhanced and the excitability of cortical neurons is increased (Pfurtscheller, 2001; Steriade et al., 1991),whilst ERS has been thought that it reflects a deactivated cortical state with reduced information processing or none at all and decreased cortical excitability(Neuper &Pfurtscheller, 2001; Pfurtscheller, 1992). However, the knowledge about the ERS has been accumulated such that the meaning of ERS is more than the state of decreased cortical excitability. The inhibitory activity of ERS can play a functional role to accentuate a task-related information processing by inhibiting other cortical areas and/or to deactivate some cortical network depending on a task context/situation (Neuper &Pfurtscheller, 2001;Hummel et al., 2002, 2006; Suffczynski et al., 1999).
Coherence refers to correlation between two sets of time-series in frequency domain. Given a cross-spectral density matrix by two time series (i.e., EEG data from two electrodes), coherence is obtained by the ratio of cross-spectral to spectral of each time series, whichindicates the degree of relative synchrony between the two time series, as shown below:
where Sii(ƒ) and Sjj(ƒ) refers to frequency spectral of electrode i and j at frequency ƒ, respectively, and Sij(ƒ) refers to cross-spectral of electrode i and j at ƒ. Higher coherence means higher degree of coupling between the two EEGs, which is interpreted as functional connectivity between two cortical sites. This idea of coherence has been extended for taking into consideration about the effect of temporal relationship between two time series to examine if one time series has influence on the other, and vice versa. This is termed directed coherence (Saito & Harashima, 1981;Kamitake, et al., 1984 cited in Wang & Takigawa, 1992). Analyzing the time series data with respect to a task event can reveal the change in the directed coherence over time with respect to the event (event-related directed coherence: EvDirCoh). By applying the EvDirCoh analysis to a set of EEG data over the cortex, functional connectivity or communication between different cortical sites and the direction of the communication for perceptual-motor process to execute a task performance can be investigated.
For these two analyses, the author and his colleagues analyzed ERSP in the reaching-grasping and the size-estimation performances (Katsumata, et al., 2009). Given the findings by the previous analysis that revealed the cortical activation pattern for the task execution, the present study conducts the EvDirCoh analysis for the data to investigate the cortical communication across different sites. Thereby, it is attempted to capture the brain dynamics characteristic to the perceptual-motor process for the task execution. In this chapter, experimental and analytical methodology for both of ERSP and EvDirCoh and those results are reported, and the dynamics of cortical activation is discussed in terms of the association of cognitive aspect with respect to the perceptual-motor process.
10 healthy participantsvolunteered for the experiment (seven males and three females with an average age of 29 ± 6.7 years).Their preferred hands for performing task movements were right hand and they were assessed as being right-handed by the Edinburgh inventory. All procedures were approved by an ethics committee. Each participant signed an informed consent form after the experimenter explained the purpose and procedure of the experiment.
2.2. Task and task conditions
Two types of perceptual-motor tasks were examined (Figure 1): (1) the participants reached out with their right hands to a target object displayed on the computer screen and touched it so as to grasp it with a pinch grip produced by the index finger-thumb aperture (Grasping) and (2) they produced the pinch grip as in Grasping task but without the reaching motion, so as to match the index-thumb aperture size with respect to the target object size (Matching).
Two different figures were usedas the target objects: (1) a single circle with a diameter of 3 cm (Neutral figure) and (2) the Ebbinghaus figure, consisting of a center circle with a 3 cm diameter surrounded by five circles with 5 cm diameters. In Matching, the participants were instructed to estimate the size of the center circle and show it with the index finger-thumb aperture size. The aperture motion was the same as in Grasping but without reaching. The aperture size against the Ebbinghaus figure was produced the with respect to the center circle. Approximately one second after holding the aperture configuration, they terminated the task movement (Figure 2 for the time course of the task paradigm).
The aperture movementwas measured in terms of the angular excursion of the metacarpophalangeal joint by attaching a Goniometer (DKH, Tokyo, Japan) to the index finger. To this end, casts were attached to the proximal and distal interphalangeal joints of index finger and thumb. Thereby, the motions of these joints were constrained such that the aperture size was produced by only the movement of the index finger-metacarpophalangeal joint. This measure of the joint excursion was used to examine the grasping aperture. A 64-channel data collection system (ESI-64 Channel System, Neuroscan, Charlotte, NC) was used to collect Electroencephalogram (EEG). The visual display of the target figure, a beep sound to cue the participant to initiate the task movement, a trigger pulse to synchronize the Goniometer data with EEG data were operated by a data collection software (LabView, National Instruments, Austin, TX)
Preliminary to the data collection, the effect of the Ebbinghaus figure for each participant was tested by the method of limit. By verbally judgingthe comparisonbetween the size of a center circle of the Ebbinghaus figure with a comparison object of a single circle with different sizes, the perceptual threshold for detecting the size difference was examined (mean and standard deviation: 2.8±0.15 cm, as opposed to 3 cm of the center circle diameter of the Ebbinghaus figure). A t-test confirmedthat the participants visually perceived the Ebbinghaus figure to be smaller than the comparison circle (p < 0.01).
The procedure of data collection was as follows(Figure 2). At the beginning of each trial, a “+” symbol was displayed at the center of the screen as avisual fixationpoint. After the participant clicked a computer mouse with their left hands, the fixation point disappeared. 2000 ms after the fixation offset, the target figure was shown at the center of the display. 2000 ms after thetarget onset, a beep sound was produced to cue the participants to initiate the task movements. 5000 ms after the auditory cue, the target figure disappeared,and it was enough time for the movement to be completed.Thereafter, the fixation point appeared for next trial. In instructing a task procedure to the participants, it was emphasized that the task was not for testing a reaction time nor a speed of task movement (The mean time of the movement initiation after the beep: 502±203 ms).The 80 trials of task movements for each condition were divided into two blocks consisting of 40 trials and performed in series. The order of Grasping and Matching was counterbalanced across the participants. The order of the figures across trials was randomized within each set of trials. They could take a few minutes break between blocks, and they could also take an inter-trial interval, during which they could blink. By these brakes, the participants could complete the whole set of trials without getting too fatigued. The whole data collection process lasted for approximately 90 minutes for each participant.
2.5. Data collection and reduction
The movement of index finger metacarpophalangeal joint was recorded (400Hz) by the Goniometer and a second order band-pass filter with a cutoff frequency of 5 Hz was used for smoothing the data. The angular velocity of the joint was obtained by numerical differentiation and smoothed by a second order band-pass filter (cutoff frequency of 5 Hz).EEG was collected from 64 scalp electrodes of the international 10-20 system referenced to the left earlobe (AC-mode, a sampling rate of 1000 Hz, a gain of 500, and a pass-band of 0.05-100 Hz). All electrodes were required a resistance of less than 2 Ω. To detect horizontal and lateral eye movements as well as blinks, electrooculography (EOG) of the right eye was collected. The data sets of EEG, joint movements, and auditory beeps were stored in the hard-drive of a desktop PC for off-line analysis.In the analysis, the EEG data was down-sampled to 300 Hz to conserve the memory of the PC and to save time consuming for calculating the coherence for each time-window within each frequency band. Failed trials due to initiating the movement before the auditory cue were eliminated from the analysis. The trials with an eye blink and noisy EEG data were also eliminated through visual inspection of EEG data profiles. EEG data was investigated with respect to the moments of the target onset and the initiation of the joint motion. To this end, EEG data sets for each trial were epoched from 500 ms before to 1000 ms after the target onset as well as after the initiation of the joint motion. Given the time from the movement onset to the maximum aperture of626±198 msec, this time window was enough to cover the movement duration to produce the aperture configuration.For analysing EEG data with respect to specific frequency components, following frequency bands were used, delta: 0.5-4 Hz, theta: 4-8 Hz, alpha: 8-13 Hz, beta: 13-30 Hz, gamma: 30-45 Hz, and higher gamma: 45Hz-100Hz.
2.6.1. Kinematics of task performances
While reaching to grasp an object, the maximum aperture by the index finger and thumb is linearly related to the object’s size (Jeannerod, 1981, 1984). Since this maximum preshape aperture is formed well before the hand has any contact with the object, this measure has been interpreted as reflecting the size estimate used in the perceptual-motor process in the prehensile activities. Based on this finding, the maximum aperture has been used as a dependent variable in many studies to investigate the influences of visual illusions on grasping (e.g., Haffenden & Goodale, 1998; Westwood,et al, 2000a; Franz et al., 2001).Because of this, the peak joint angle measured by the Goniometer was used as the measure of the maximum aperture for the prehensile movement. The time of the joint movement initiation was determined by the start of flexion movement of the metacarpophalangeal joint,at which the velocity of the joint kinematics started to show a positive value. This measure was used to epoch the EEG with respect to the onset of task movement.
2.6.2. Analysis of the event-related spectral perturbation
The event-related spectral perturbation (ERSP) analysis was conducted by using a toolbox with graphic interface, EEGLAB, that is operated under the MATLAB environment (Delorme & Makeig, 2004). The epoched window of 1500 msec in a single trial was divided into brief subwindows of 214 msec with a sliding latency of 3.3 msec, corresponding to 98 % overlapping between the successive subwindows. Wavelet analysis using sinusoidal transform was conducted for each of the subwindowed-epochs, and itobtained power spectrum estimates ranging from 0.59 Hz to 99.6 Hz with a frequency increment of 0.59 Hz. The power spectra over the sliding latency windows were computed for each trial, and normalized by subtracting baseline spectrum from each spectrum estimate. The baseline spectra were obtained by computing the mean spectra of the EEG data windowed for 500 msec before the moment of the target onset. Mean ERSP was obtained on each participant basis by computing the average of the baseline-normalized ERSP across all trials. To capture visually the global picture of ERSP, the grand average ERSP across the participants was plotted on the 3Dtime-frequency space (a spectrogram) on which the power spectrum of each frequency (in dB) at each sliding time-window was indicated by a colored surface.
The significance of increase/decrease in the power spectra with respect to the baseline spectra was examined for the mean ERSP in each time-frequency component of the spectrogram on each participant basis.To this end, non-parametric tests (a two-tailed Wilcoxon signed-rank test) was conducted with the null-hypothesis of no difference between the baseline spectra and spectra of each time-frequency components (i.e., ERSP value is zero) and the significant level of p<.05. This test was conducted for all of the time-frequency components.The difference in ERSP pattern between Grasping and Matching conditions was examined by the two-tailed Wilcoxon signed-rank test (p<.05) for each of time-frequency components. The results of z-values, which revealed the significant “Grasping-Matching” differences, for each time-frequency component, were plotted on a time-frequency plane. This gives the general view about the distribution of the significant ERS/ERD difference between the task conditions over the cortex.
In the previous study by the author and his colleagues (Katsumata et al., 2009), the ERSP analysis described above was conducted for 55 electrodes over the cortex with further analysis to summarize the characteristics of those ERSP profiles.For reporting the results in this chapter, ERSPs of the electrodes on the left-hemisphere were focused.
2.7. Analysis of event-related directed coherence
Event-related directed coherence (EvDirCoh) was calculated by the program operated under the MATLAB environment, which was developed by Takahasi, Baccalá and Sameshima (2008). For the calculation, the EEG time series were detrended before coherence calculation, Nuttall-Strand algorithm was used for estimating multivariate autoregressive models (MAR), and Akaike information criterion (AIC) was used for the criterion to choose theMAR order.The coherence was calculated with the time-window of 60ms with 30 ms increment, resulting in 15 time-windows from 500 ms to 1000 ms after the onset of target figure and task movement, and with respect to each of frequency bins. The time windows and frequency bins were set coarser than those used for the ERSP analyzes, since executing the program for calculating directed coherence was much more time consuming, given the number of electrode combinations for calculating the coherence. However, the preliminary analysis, for some of the data, using the finer time windows and frequency bins confirmed that this does not lose information about the global picture of coherence pattern over time and across frequencies.
Mean EvrDirCoh across trials within each participant was obtained for Grasping and Matching conditions respectively and subjected to a two-tailed Wilcoxon signed-rank test if the event-related coherence was significantly different from the baseline coherence (p<.05). This test was conducted for each of the time-frequency components. The baseline coherence was obtained by the mean coherencefor the time windowof 500 msec before the moment of the target onset. Only the EvrDirCohs, which were significantly higher than the baseline coherence, were subjected for reporting the results. A set of significant EvrDirCoh across all the time-windows and frequency bins were plotted on a time-frequency plane, which shows how coherence within a particularfrequency band increased or decreased over time with respect to the onset of target figure and/or task movement (Figure 3).
Calculating the coherence of all the electrodes available was redundant. Even in the left-hemisphere, there were 24 electrodes leading to 23 time-frequency plots in one direction of EvDirCoh and another 23 plots for the other direction. Therefore, the present study focused on the electrodes in the left-hemisphere, which cover the cortical sites associated with the dorsal/ventral stream and visuomotor process in a hand movement against a visual target (O1, PO3, PO7, P3, P7, CP3, TP7, C3, T7, FC3, FT7, F3, and F7), but it still lead to 299 plots for coherence in the one direction and another 299 for the other direction. Since the primary aim of the study was to capture the global characteristics of EvDirCoh pattern over the cortex, it was attempted to summarize the results of the EvDirCoh plots in the following procedures.
2.7.1. Electrode combinations showing marked EvrDirCoh
To capture what electrode combinations revealed marked EvrDirCoh, the rate of significant EvrDirCoh was calculated for each time-frequency plot. To this end, the number of time-frequency components on a time-frequency plane within a particular time-window, which showed the significant EvrDirCoh, was divided by the total number of time-frequency components within the corresponding time-window, and multiplied by 100 to show the result in percent. The ranges of time window were: (1) 500 ms from the target onset; (2) 500 ms before the onset of task movement to 100 ms after the onset; and (3) 100 ms after the movement onset. The same calculation was done for EvrDirCoh in the other direction. In the figure, the higher value means that more time-frequency components revealed the significant EvrDirCoh.
2.7.2. Pattern of change in EvrDirCoh over time
To capture the change of EvrDirCoh over time with respect to the onset of target figure and task movement, the significant EvrDirCoh values within each time window across all the frequency bins, were summed up respectively, and plotted over time. The reason for summing up the values rather than obtaining mean was to accentuate visually the overall change of EvrDirCoh pattern that appeared on the time-frequency plane
2.7.3. Frequency band showing marked EvrDirCoh
To capture what frequency band revealed marked EvrDirCoh, the rate of significant EvrDirCoh was calculated by the number of time-frequency components, within the frequency band, showing the significant EvrDirCoh, divided by the total number of time-frequency components within the corresponding frequency band. The rate was obtained for each of the focused electrode by all the time-frequency plots across all the electrode combinations.
3.1. The maximum aperture
The perception about the size of the target figure was examined in terms of the maximum angle of the index finger metacarpophalangeal joint during the aperture motion. A 2×2 repeated-measure ANOVA with the main effects of the tasks (Grasping and Matching) and the objects (Neutral figure and Ebbinghaus figure) revealed the significant main effects of the task (F(1, 9) = 5.098, p =.050) and the target figure (F(1, 9) = 6.089, p =.036) with no interaction effect between them (F(1, 9) = 0.112, p =.75). The maximum aperture angle was smaller for the Ebbinghaus figure than for the Neutral figure and smaller in Grasping than in Matching (Figure 4).
3.2. ERS/ERD with respect to the target onset
Figure 5-a shows exemplary spectrograms in F3, FC3, C3, CP3, P3, PO3, and O1 with respect to the target onset in Grasping performed to Neutral figure. According to the change in ERSP over time in the figures, increase of power spectra (ERS) within a lower frequency band (delta- and theta-bands) appeared immediately after the target onset. The decrease of the spectra (ERD) around the alpha-band frequency was also observed. This ERS-ERD pattern was mainly distributed in the parietal and occipital regions (O1,PO3, and P3 in Figure 5-a), than in frontal regions (F3, FC3 and C3 in Figure 5-a). These ERS and ERD were significantly different from the baseline spectra. According to the analysis conducted in the previous study by the author and his colleagues (Katsumata et al., 2009: Table 1), the timing of these ERSP with respect to the target onset was 74±32 ms in the delta-ERS and 81±26 ms in the theta-RES, which were followed by the alpha- and beta-ERD (215±59 ms and 148±30 ms, respectively).
3.3.ERS/ERD. with respect to the movement onset
Exemplary spectrograms with respect to the movement onset in Grasping and Matching to the Neutral figure were shown in Figure 5-b and 5-c respectively. Before the movement onset, the lower frequency ERS were observed in the frontal and central regions. Prominent ERD at around the alpha-band was observed in the central and parietal regions before and during the movement in both of Grasping and Matching. The prominent feature in Grasping was the gamma-band ERS, which appeared before the movement onset in the parietal and occipital regions and continued during the movement execution. As opposed to it, in Matching, the higher frequency power spectra did not reveal any change during the movement. According to the previous study(Katsumata et al., 2009: Table 1), the ERD started to increase, on average, 129±88ms in the central-parietal regions and 78±52ms in the frontal region before the movement onset and the ERS occurred 214±9 ms before the movement onset. The comparison of ERSP betweenGrasping and Matching revealed the significant difference in the parietal-occipital regions at the gamma-band (Figure 5-d).
In summary, both of Grasping and Matching performances were affected by the illusory object, according to the aperture joint angle. In execution of the task performances, the alpha-wave ERD were observed and it did not show a notable difference between Grasping and Matching. The task-dependent difference was the gamma-wave ERS that was observed in Grasping but not in Matching.
3.4. Electrode combinations showing the marked coherence (EvDirCoh)
The rate of significant EvDirCoh on each time-frequency plane across electrode combinations was visualized in Figure 7 to Figure 9. Based on these figures,the results of electrode combinations that showed the characteristics of remarked coherences were reported as below. Refer to Figure 6 for the electrode locations on the left-hemisphere, which correspond to the location of each cell on the plot. Figure 7 shows the rate of significant EvDirCoh after the target onset, and Figure 8 and Figure 9 show the rate before and after the movement onset (i.e., the movement preparation and execution phase), respectively.
3.4.1. The occipital region (O1) in response to the target onset and during the movement production
As appeared in O1, prominent EvDirCoh were observed after the onset of target figured, which implies response to the visual input about the target. The coherences between O1 and other sites were more prominent in response to the visual target, after which those were restricted to a few electrodes. In Grasping, the coherence from O1 to the occipito-parietal/inferior parietal region (CP1/CP3/CP5/P1/P3/ PO3/PO5/P5) was observed after the target onset and lasted for the movement execution. EvDirCohto T7and P7 (the ventral stream) was observed, and EvDirCoh to P7 lasted for the movement execution. EvDirCoh from the inferior-parietal/superior-temporal region (PO5/P5/CP5) was observed. EvDirCoh in Matching appeared to be less remarkable, compared to those in Grasping, except for EvDirCoh to CP5 and P1/PO3 after the target onset and from PO5/PO3 after the target onset and during the movement.
3.4.2. Electrodes relevant to the dorsal stream of visual processing (PO3, P3, and CP3)
After the target onset, in Grasping, the prominent EvDirCohs were observed in P3, CP3, and PO3, which were from the divergent areas including the inferior-frontal region (F7/FT7), the medial-central region (FC1/FC3/C1/C3), the temporal region (T7/CP5), and the parieto-occipital region (PO3/PO5). Before the movement onset, EvDirCoh to PO3/P3 from the frontal, temporal, and parietal regions became more prominent, and these frontal/temporal EvDirCoh lasted for the movement execution. In Matching, EvDirCoh from PO3 to the parietal-occipital and posterior-temporal regions, as well as the coherence from the occipito-parietal region (O1/PO3/PO5) to P3/CP5 appeared to be remarkable. Before the movement onset, EvDirCoh from PO3 to the divergent areas, particularly the central-parietal region, became prominent, and lasted for the movement execution. EvDirCoh from the frontal region to CP3 and EvDirCoh from the parieto-occipital region to CP3/P3 were also remarkable.
3.4.3. Electrodes relevant to the ventral stream of visual processing (PO7, P7, TP7, and T7)
After the target onset, PO7/P7/PT7/T7 in Grasping showed EvDirCoh to electrodes in different locations and P7 and T7 received EvDirCoh from their neighborhoods, FT7/T7/PT7 or CP5/P5, respectively. Furthermore, during the movement execution, EvDirCoh from T7 to the frontal, central, and parietal regions were pronounced. EvDirCoh to T7 form the frontal region (F5/FC5) was also remarked.As for Matching, PO7 showed the pronounced EvDirCoh to the central and parietal regions after the target onset. Whilst these EvDirCohs became more prominent and extended to the frontal region before the movement was initiated, PO7 also showed the coherence from the frontal and the parieto-occipital region. This PO7-to-central/parietal and the frontal-to-PO7 EvDirCoh lasted during the task execution.
3.4.4. Electrodes located on the frontal region (F7, F3, FC3, and FT7)
After the target onset, EvDirCoh from F7/FT7/FC3 to posterior temporal region (PO7/P7) was observed in Grasping. Matching did not show such prominent EvDirCoh as in Grasping, except for EvDirCoh from FC3 to the parieto-occipital region (PO7/PO5/PO3). Before the movement was initiated, FT7 and F7 in Matching showed the remarked EvDirCoh to the central-parietal region, whilst the F7 in Grasping received EvDirCoh from FC5/C5 and PO5/P5. FC3 showed EvDirCoh to/from divergent areas. During the movement execution, EvDirCoh to the central-parietal region, which were centered the medial region, were observed in Grasping. FC3 in Grasping showed the prominent EvDirCoh from the frontal, central and temporal regions. EvDirCoh in Matching during the movement execution were less remarked than in Grasping, except for EvDirCoh from FC3 to C1/C3/CP1/PO3.
3.4.5. The electrode located on the motor cortex for movement production (C3)
The focus of coherence in C3 is on the preparation and execution of task movements. The remarkable feature before the movement initiation was that C3 received EvDirCoh from divergent areas in Grasping, which contrasts to EvDirCoh in Matching mainly from the frontal region. During the movement, EvDirCoh to C3 in Grasping became pronounced and centred in the frontal and temporal regions.
3.5. Coherence profile over time
According to Figure 10, all most all of the analyzed electrodes showed increase in the coherence after the target onset, which lasted for 500-600 ms. This indicates that there were communications across different cortical sites, possibly for processing information about the target figure for the up-coming task execution. Regardless of task conditions (Grasping or Matching), the electrodes that are associated with either of the dorsal or ventral stream showed the increase of coherence, which indicates that both of the streams might be involved in the visual processing about the target figure. The pattern of coherence over time with respect to the onset of task movement was as follows.
O1 that corresponds to the visual cortex in the left-hemisphere showed the increase in some of coherences before the initiation of the movement and lasted during the movement execution. This was remarked in Grasping, compared to those in Matching. Before the movement onset in Grasping, increase of coherence was observed in PO7, P7, FT7, CP3, FC3, and F7. This was followed by the coherence increase in TP7, P3, and T7 at the movement onset. The increase of coherence was also observed before the maximum aperture was produced (e.g., T7, CP3, C3, FC3, and F3), and after the peak aperture (e.g., PO3, PO7, P3, TP7, T7, FT7, and CP3). According to the timing of these coherences, there seem to be temporal pattern of coherence change from the preparation of movement to the movement completion. For instance, the increase of coherence in PO7 and P7 before the movement onset was followed by TP7and T7 at around the movement onset, and FT7 after the peak aperture production (the 1st column of Figure 11), which seems to reflect the activity of the ventral stream. Likewise, the increase of coherence in PO3 before the movement onset was followed by P3 at around the onset, and CP3 after the peak aperture production (the 2nd column of Figure 11), which seems to reflect the activity of the dorsal stream.
As for Matching, according to the appearance about Figure 11, there were less remarked increases of coherences, compared to those in Grasping. The remarked increase of coherence in O1 was observed after the movement onset. Before the initiation of movement, the increase of coherence was observed in PO3, PO7, T7, FT7, C3, FC3, F3, and F7. From the movement onset to the moment of the peak aperture, coherence of PO3, PO7, P3, CP3, C3, and FC3 showed remarked increase. After the peak aperture, the increase of coherence was observed in PO3, PO7, P3, TP7, T7, and CP3. These coherence patterns indicate that Matching movement was executed with less connections or communications among the cortical sites, compared to Grasping.
3.6. Frequency band showing the remarked coherence
Figure 12 shows the rate of significant EvDirCoh within each frequency band. In this figure, higher value indicates that more time-frequency components revealed significant EvDirCohwithin corresponding frequency band. The figure can also indicate what frequency band was dominant across all the frequency bands.
With respect to the target onset, the lower frequency bands (delta, theta, and alpha) appeared to be more dominant compared to the beta, gamma, and higher gamma bands in the most of electrodes. Some of the electrodes in Matching(e.g., PO3 and PO7) revealed higher rate of significant time-frequency components in higher frequency bands. As opposed to it, with respect to the movement onset, the rate of higher frequency band (gamma and/or higher gamma) increased, which was particularly prominent in CP3, C3, FC3, PO3, PO7,P3, P7, and T7 in Grasping and/or Matching.
4.1. The size-illusion effect for Grasping and Matching
The comparison of task performances in terms of the peak aperture angle showed a significant difference between the tasks (Grasping and Matching) as well as between the task-objects (Neutral and Ebbinghaus figure). This indicated that not only Matching but also Grasping was affected by the size illusion. This implies that the cognitive processing about the size perception was also involved in the performances in Grasping. The Grasping-Matching difference in the illusion effect can imply the different magnitude of the illusion effect and/or the task-dependent difference in scaling relationship between the perceived object size and the produced aperture size. In other words, the cognitive processing for shaping the aperture based on the size perception may not be exactly the same in these two task executions.
The effect of visual illusion on a prehensile movement task to the illusory object as well as on a perceptual discrimination task have also reported in some earlier studies (e.g., Franz et al., 2000, 2001; Glover & Dixon, 2002; Mendoza et al., 2005), which contradicts to the hypothesis of dissociative systems for perception and action by the ventral/ dorsal stream (Aglioti et al., 1995; Goodale& Jakobson, 1992; Haffenden & Goodale, 1998). These illusion effects are attributed to the task conditions, such as, complex target figures (McIntosh et al., 2004), movement production in an open-loop manner (i.e., movements to a remembered object: Heath et al., 2005; Westwood et al., 2000a), and online reorganization of the grasping aperture in response to the change of object size (Heath et al., 2006b). These findings have led to arguments such that: the motor system integrates the ventral and dorsal streams (Ellis et al., 1999); the sensorimotor system can operate independently of the cognitive/perceptual system (Flanagan & Beltzner, 2000); a prehensile movement is produced through multiple visuomotor processes (Westwood et al., 2000b); the ventral stream has connection to the prefrontal areas and, thereby, it may associate with the visuomotor control of a prehensile action (Lee & van Donkelaar, 2002); and online prehensile control may be produced through egocentric and allocentric visual information processing (Heath et al., 2006a).
Even though there are functionally and anatomically distinctive visual pathways at the cortical level, executing a visuomotor task may be achieved through an integrative process of these pathways. These visual pathways may be involved differentially in task-dependent manner. For instance, if a pure visual discrimination task such that participants are asked to discern which of two objects looks larger/smaller, it is expected that a cortical activity associated with the ventral stream may be dominant (Farrer et al., 2002; Westwod et al., 2000b). Likewise, if reaching or grasping needs to be produced in an open-loop manner (e.g., a visual object is occluded before or right after the onset of a prehensile movement), the movement production needs to be based on the memory of the object shape (Heath et al., 2005; Westwood et al., 2000a; Westwood et al., 2000b). In these cases, the nature of task performance requires cognitive processing, which will be achieved via the ventral stream dominantly. Given the above results about the Ebbinghaus effect, cortical activation patterns revealed in ERSP and EvDirCoh need to be interpreted in terms of how cortical sites associated with the dorsal/ventral stream in an integrated fashion. In this sense, the difference in the ERSP and/or EvDirCoh pattern between Grasping and Matching may reflect the qualitative difference in the integrated cortical process with which cognitive processing is associated.
4.2. Comparison of ERSP pattern between the task conditions
4.2.1. ERD observed in Grasping and Matching
ERD has been regarded as representing an activated cortical state and thereby sensory, motor, and/or cognitive processing is enhanced. Therefore, the cortical regions showing ERD during the task performances can be regarded as functionally associating with the perceptual-motor process for the tasks. The comparison of the ERSP patterns between Grasping and Matchingrevealed that the ERD pattern appeared to be virtually identical. It indicates that the cortical excitability assessed by ERD during the two perceptual-motor tasks was not distinctive. This lack of distinction in ERD between Grasping and Matching with the size-illusion effect in both the task performances suggests that the perceptual-motor process in Grasping, somehow, involved cognitive processing about the target figure as in Matching.
The ERD after the target onset were observed not only in the occipital region (i.e., visual area), but also in the central-parietal regions. This implies some visual processing rather than just responding to the visual stimulus, which could possibly be associated with the task execution. The ERD in occipital-parietal regions was observed with respect to the aperture movement onset as well as after the target onset. Based on that the posterior parietal cortex functions for the integration of the sensory information as well as the visuomotor coordination (Buneo & Andersen, 2006; Burnod et al., 1999; Culham et al., 2006; Darling et al., 2007; Stein, 1995), the occipital-parietal ERD can be interpreted that the visual information was further processed via the higher level of visual areas toward the movement execution.
The ERD in F3, F4, FC3, FC4, C3 and C4 in Figure 5 indicates that the frontal and pre-frontal areas were involved in the task execution. This frontal area ERD was observed not only in Matching but also Grasping, which indicates that the frontal cortical network may be involved in the execution of both tasks. The functional connection between the frontal area and the parietal cortex plays a role for the spatial organization of arm and hand motions, such as, reaching and grasping (Caminiti et al., 1998; Rizzolatti et al., 1997). The premotor and motor cortices function for preparing and controlling movements to achieve a given motor task goal (Kandel et al., 2000; Zigmond et al., 1999). The frontal ERD seem to indicate these functional activities for the task execution.
4.2.2. ERS as the characteristic of Grasping
The notable feature in the ERSP spectrogram was the ERS observed not in Matching but in Grasping. The ERS within the higher frequency band, corresponding to the gamma-wave, started before the onset of the aperture joint motion and continued during the Grasping execution.The ERS was observed mainly in the occipital-parietal regions (e.g., O1, O2, OP3, OP4, P3, P4 and CP3 in Figure 5), those of which associate with sensory information processing and motor production.Originally, ERS has been regarded as the cortical state of decreased excitability and suppressed processing of specific information in underlying neuronal networks (Neuper &Pfurtscheller, 2001; Pfurtscheller, 1992). However, ERS has been observed in association with ERD and the accumulation of findings about ERS by recent studies, as described below, suggests that not only ERD but also ERS plays a role in the cortical network’s function for the execution of task performances. The cortical activity pattern of ERS in association with ERD was observed, such as: The locally specific alpha- and/or beta-ERD accompanied by ERS in neighboring areas (the focal ERD/surround ERS: Gerloff et al., 1998; Neuper &Pfurtscheller, 2001; Pfurtscheller &Neuper, 1994); The mu-rhythm ERD in the motor cortex with respect to the movement onset followed by the beta-wave ERS with respect to the movement offset (Pfurtscheller et al., 1999b); and a strongermu-rhythm ERD over the motor cortex accompanied by a smaller beta-wave ERS during manipulation of a cube (Neuper et al., 2006). Amongst those findings, the inhibitory mechanism of ERS, which is for limiting or controlling excitatory process associated with task execution, has been proposed, such as, the antagonistic ERS-ERD as a mechanism to enhance focal activation by simultaneous inhibition of the other cortical areas (Neuper &Pfurtscheller, 2001; Suffczynski et al., 1999) and the post-movement ERS as reflecting some selective attention or an active inhibition to accentuate a task-related information processing by inhibiting the task irrelevant other cortical areas (Neuper &Pfurtscheller, 2001). When a learned movement pattern was intentionally prevented from being retrieved, increase in the alpha-band task-related power (i.e., alpha-wave ERS) was observed in the primary motor cortex that indicating the inhibition (Hummel & Gerloff, 2002). ERS observed in memory process was interpreted as an inhibitory role to block a memory search from entering irrelevant parts of neural networks (Klimesch, 1996).
Given the above findings, the gamma band ERS pattern during Grasping can be regarded as representing the aspect of a perceptual-motor process, which is characteristic to Grasping. Whilst the alpha and lower beta ERS represent reduced information processing or deactivation in underlying cortical networks, the gamma band ERS has been reported during sensory encoding (Galambos et al., 1981), cognitive processing (Basar & Bullock, 1992), and movement execution (DeFrance & Sheer, 1988). These gamma band oscillations were thought to be related to binding of sensory information and integrating sensorimotor process, since binding signals at a higher level of processing may require signal transmission higher than the alpha and lower beta oscillation (Singer, 1993; Pfurtscheller, 1998). In this sense, the gamma band ERS may represent a functional mechanism beyond being inhibitory.
In this analysis, the two oscillatory components, one showing ERD and the other ERS, were observed in a single electrode in the parietal-occipital regions. Different frequency oscillations embedded at a single electrode are interpreted as one neural network generating different types of oscillations (Pfurtscheller & Lopes da Silva, 1999). Therefore, the ERD in both of Grasping and Matching and the ERS observed only in Grasping seem to be associated with different functional roles respectively. Considering about the nature of task execution in Grasping and Matching, the ERD pattern may represent the functional aspect common to the both of Grasping and Matching (i.e., associating the index-thumb joint motions with respect to the perception about the size of visual target), and the ERS pattern may represent the functional aspect exclusive to a visually-guided aperture organization process in Grasping (e.g., continuously updating the target object-hand spatial relationship to reach out and shaping the aperture).
4.3. Cortical communication in terms of EvDirCoh and the nature of task execution process
According to the feature of the rate of significant EvDirCoh, which captures the pattern of coherence appeared on the time-frequency plane, the following characteristic EvDirCoh was observed in each of the task performances. The coherence analysis has been applied to time series of neural system based on the assumption that the coherence reflects the functional connectivity, neural communication and/or signal transmission between two or more neural activities (Sameshima & Baccalá, 1999). The results in the present study are interpreted from this perspective.
After the target onset, the significant coherences were dominant amongst the lower frequency (the delta, theta, and alpha bands in Figure 12). This implies that the visual processing in response to the target object was achieved by the neural communication with those frequency bands. As opposed to it, the gamma and higher gamma bands became prominent after the movement onset. Those prominent frequency bands were observed particularly in CP3, C3, FC3, PO7, P3, P7, and T7 in Grasping and Matching. Since these sites are associated with the coordinate transformation and the hand movement organization, the result implies that the communication between different cortical sites occur at higher neural oscillation during the preparation and execution of task movement.
As for the temporal characteristics of EvDirCoh, the most of the analyzed electrodes showed increase in EvDirCoh after the target onset and lasted for approximately 500 ms. This EvDirCoh increase may reflect the communications across different cortical sites for processing information about the target figure for the up-coming task execution. With respect to the movement execution in Grasping, the increase of significant EvDirCoh was observed with the temporal order of:(1) PO3, PO7, P7, FT7, CP3, and FC3 before the movement onset;(2) P3, TP7, and T7 at round the moment of the movement onset;(3) T7, CP3, C3, FC3, and F3 during the movement execution; and (4) PO3, PO7, P3, P7, T7, FT7, CP3, C3, and FC3 after the peak aperture produced. After the movement onset and before the peak aperture production, the dorsal stream-relevant electrodes (P3 and CP3), the ventral stream-relevant electrodes (PO7, P7, T7, and FT7), the frontal region (FC3 and F3), and the motor cortex (C3) showed the increase of the coherence. This may reflect the movement execution by the cortical communication for exchanging information. These temporal patterns were less pronounced in Matching, even though there were increases of EvDirCoh amongst the electrodes. In both Grasping and Matching, the involvement of the dorsal/ventral stream and the frontal cortex in the preparatory and execution phases was shown in the temporally ordered fashion.
4.3.1. Coherence relevant to the task events (the target onset and the movement preparation-execution)
After the target onset, the visual cortex (O1) and other areas showed coherence to/from different cortical site. This may implies the response to the visual target. Particularly, the parietal-occipital region (PO3 and PO7) in Matching showed the prominent coherence to the parietal, occipital, and posterior temporal regions. This means that there were functional connections within these regions, which can be possibly associated with the visual information transmission from the visual cortex to the higher level of visual processing. Since executing the Matching task requires estimating the center circle size, which was potentially achieved by combining the information about the surrounding circles, information processing in Matching can be more demanding than in Grasping.
In the movement preparation phase, PO7 and PO3 in Matching showed the prominent coherences as in the target onset phase. Furthermore, PO7 and CP3 showed the connection from the frontal region, whilst the frontal region showed connection to different sites. These characteristics may reflect the cortical communication for visual processing and movement preparation. As for Grasping, the prominent characteristics were the connection to the central and parietal regions (C3, CP3 and P3) from divergent areas, mainly the inferior frontal, frontal, and temporal regions. Since the parietal region plays a functional role for coordinate transfer to organize a limb movement, these coherence pattern may reflect this functional activity. Another feature was the connection from electrode associated with the ventral stream, PO7, PT7, and T7 to the central-parietal region. As described above, different EvDirCoh patterns were observed in the preparatory phase before initiating the task movement.
The remarkable feature of EvDirCoh in Grasping for executing the movement was observed in the connection to frontal-central region (FC3/C3/CP3) from inferior frontal, frontal, and temporal regions. Furthermore, the connection from the temporal (T7) and frontal (FT7/F7/F3/FC3) to the central and frontal region became more prominent than in the preparatory phase. As opposed to it, in Matching, the remarked coherence was observed in restricted sites, which were similar to that in the preparatory phase. This indicates that executing the task movement was achieved through more active cortical communications in Grasping, compared to in Matching. The movement of reaching and grasping requires the perceptual-motor process to organize a limb movement, whilst the size-mating task is executed only by producing the index finger-thumb motion. Such difference can be reflected in the P3/CP3/C3 characteristics in Grasping. Contrary to it, the process of assessing the object size and associating the perceived size with a particular finger motion is a central part of Matching, and this aspect might be reflected in the target onset and preparatory phases.
4.3.2. EvDirCoh characteristics associated with the dorsal/ventral streams
The visual information from the primary visual cortex is transmitted further via the dorsal stream to the middle temporal area and the medial superior temporal area, as well as via the ventral stream to the posterior inferior temporal cortex and the anterior inferior temporal cortex(Kandel et al., 2000; Zigmond et al., 1999). The electrodes PO3, P3, and CP3 are relevant to the dorsal stream, and PO7, P7, TP7, and T7 to the ventral stream. In the present study, EvDirCoh patterns associated with these electrodes showed following characteristics.In Grasping, PO7, P7, TP7, and particularly T7(the ventral stream) showed the marked connection mainly to the frontal and central regions during the task execution. P3 and CP3 (the dorsal stream) also showed the connection from the frontal and temporal regions, especially in the preparatory and execution phases. Since the Grasping performance was also affected by the size-illusion, this result does not contradict to the idea of the cognitive role of the ventral stream. As opposed to it, in Matching, T7 and FT7 did not show the remarked coherence,even though PO3 and PO7 revealed the marked coherence during the task execution.
These results revealed that the involvement of the dorsal and ventral streams for the task execution was not exclusive betweenMatchingandGrasping. But the patterns of EvDirCoh were different between them, which suggest that the involvement of the cortical stream and the cortical communication are task-dependent. The analysis of ERSP did not reveal the detail of cortical activation pattern as shown by EvDirCoh, even though it could demonstrate the task dependent characteristics of cortical activity. Difference between these analyses is that ERSP, as well as other analytical measure (e.g., hemodynamics in fMRI and the event-related potential in EEG) examine the amount or magnitude of neuronal actvity, whilst coherence takes care of the relationship between two or more neural activities. Therefore, information about the cortical activities, which is more than the amount, can be available. This aspect seems to be attributed to the findings reported above. The present study could reveal the characteristics of cortical activity and communication, which could be dependent on the nature of the perceptual-motor process in the task execution. Further studies are necessary for investigating what task situations/conditions can induce the ERSP/EvDirCoh characteristics and how those are associated with the involvement of cognitive processing for a perceptual-motor task.
The involvement of cognitive processing in the perceptual-motor process for achieving a motor task goal was examined by Grasping and Matching tasks, in terms of cortical activity. To this end, the event-related spectral perturbation and directed coherence were analyzed. Those analyses could reveal the different cortical activation and communication pattern, which could be attributed to the difference in the nature of perceptual-motor process in these tasks.
This study was founded by the Grant-in-Aid for Scientific Research (#20500507) in Japan Society for the Promotion of Science, and the Research Grant in Daito-Bunka University (#220102). EEG data collection was conducted by the support of Dr. Kuniyasu Imanaka and his doctoral students at Tokyo Metropolitan University.