Brain Networks of Emotional Prosody Processing in a Foreign Language Versus Mother Tongue

Abstract

Increased interest in the relationships between the brain and behavior over the past several decades has made brain network process of emotional prosody a topic of study in disciplines like neurology, psychiatry, neurolinguistics and neuroscience. Because emotional prosody has a key role to implication of timbre component, mood sense, and prosodic content. Also, it serves a highly important function for sense, the meaning to be reflected, and ability to provide effective communication. Therefore, the knowledge of how the emotional prosody sequence works in the brain will contribute to both language development and foreign language teaching as well as clinical evaluation of individuals with verbal communication difficulty. In the literature, neuroimaging and neurophysiological studies about investigating emotional prosody have produced controversial results in specifying similarities versus differences mother tongue acquisition and foreign learning neural networks. For this reason, this review study takes an interdisciplinary perspective to identify the neural networks of emotional prosody in mother tongue and foreign language learning process with different imaging modalities.

1. Introduction

Emotional prosody is taken place by marking the mood values in the language, using verbal and non-verbal languages together or separately [1]. It is stated in the literature that the effective use of emotional prosody in speech contributes to the comprehensibility of the message to be conveyed. As a matter of fact, Kimelman states that a speaker conveying his/her message in emotionless and monotonous way may affect understanding negatively and will not attract attention [2].

Early clinical studies have contributed tremendously to our present knowledge of the neural correlates of emotional prosody. In particular, lesion studies purposed to determine the respective hemisphere to emotional speech and language. The first comprehensive study known in the literature on processing of emotional prosody in the brain was conducted by Hughlings-Jackson. According to the study, it was assumed that the phonological component was processed in bilateral hemispheres of the brain. Also, the author highlighted that the prosody system, which contains emotional states, was located in the right hemisphere [3]. In a similar vein with this study, Heilman et al. reported that lesions in the right hemisphere tend to disrupt emotional speech comprehension, while split-brain patients perform verbal deficiency when speaking about emotional stimuli [4]. Later work that was effected from Hughlings-Jackson [3] and carried out by Van Lancker it was reported that the prosody reflected emotional state was localized in the right hemisphere, but phonemes contained the sound string was located in the left hemisphere [5]. Also, in a similar way with these studies, Ross pointed out that the emotional prosody system was processed only in the right hemisphere, as well [6]. This early evidence about the predominant role of right hemisphere in emotional prosody was followed by subsequent patient researches in which the relative contribution of specific right hemisphere brain areas to these functions were determined [7]. Many studies based on patient data collected during intrasurgical electrocortical stimulation show the right fronto-central operculum sensitivity for emotional prosody [8]. In addition to this, not only patient data but also recent studies made by neuroimaging techniques as well, question a strong right hemisphere lateralization of emotional speech and language function. Although some data confirm a significant role of right cortical and subcortical structures [9], a great deal of fMRI studies used single word, verify the contribution of bilateral hemispheres or even the left hemisphere only to emotional prosody and language processing. For instance, in one fMRI study conducted by Buchanan et al. it was aimed to recognize the brain localization of emotional prosody and verbal components of spoken language. As a result of the study, authors reported the bilateral brain hemisphere domination in the processing of emotional prosody system [10]. On the other hand, Luo et al. demonstrated the left mid-fusiform gyrus activation for emotional words [11].

Emotional prosody is an important communicative element, pointed as one of the main extra linguistic attributes present in oral communication. It plays a crucial role during social communicative interaction in mother tongue acquisition and also second language learning process. Due to its important role in verbal communication, it has a critical relationship within many disciplines such as linguistics, neurology, psychiatry and computer sciences. A significant increase has been observed in studies on the relationship of emotional prosody, which is related to many disciplines, with laterization, localization, activation areas and other language components in the brain [12, 13, 14, 15]. One of the main reasons of this subject may that the findings obtained from both neuroimaging and lesion studies on patients do not provide a consistent result confirming the right-hemispheric laterization of emotional prosody. As a matter of fact, in one comprehensive meta-analysis study on emotional prosody carried out by Kotz et al., it was stated that emotional valence is predominantly lateralized in the right hemisphere in studies made with neuroimaging techniques, but it was also stated that bilateral or left hemisphere laterization in studies conducted with electrophysiological methods [15].

Emotional prosody has a significant importance to implication of timbre component, mood sense, and prosodic content. Also, it serves a highly important function for sense, the meaning to be reflected, and ability to provide effective communication. Therefore, it is thought that the knowledge of how the emotional prosody sequence works in the brain may contribute to both mother tongue acquisition and foreign language teaching. In the literature, neuroimaging and neurophysiological studies about investigating emotional prosody have produced controversial results in specifying similarities versus differences in mother tongue acquisition and foreign learning neural networks. For this reason, this review study aims to present an interdisciplinary perspective to identify the neural networks of emotional prosody in mother tongue and foreign language learning process with different imaging modalities.

2. Brain networks of emotional prosody in mother tongue acquisition

The researches towards to determine the brain localization of emotional prosody in mother tongue acquisition were generally studied to be in patients with right hemisphere damage [16, 17]. It is thought that this can be due to the view that emotional function appearances of the language are localized in the right hemisphere. In the majority of studies this view was generally supported, and it was pointed out the right-hemisphere role in detail for expression of emotion through prosody. As a matter of fact, in one comprehensive review study scanned researches that carried out patients with right hemisphere damage which was conducted by Mitchell and Crow it was demonstrated some deterioration in the basic functioning structures such as perception of emotional states in the mother language [18]. Likewise, in another study carried out by Brådvik et al. and conducted on Swedish native language speakers with right hemisphere damage, it was showed that participants did not able to analyze the emotions. Authors determined the lower part of right hemisphere role to comment this result [19]. Also, Patel et al. , investigated a consecutive series of 41 English native speaker patients with acute ischemic right hemisphere. As a consequence of the study, authors reported two novel and important results. First, they identified abnormal patterns of acoustic features that contribute to diminished emotional expression of right hemisphere stroke survivors, as rated by healthy listeners. As the second, authors stated that the measures of acoustic features associated with impaired expression of emotion were associated with lesion load in right inferior frontal gyrus, pars opercularis or supramarginal gyrus, or associated white matter tracts, particularly inferior fronto-occipital fasciculus, superior longitudinal fasciculus, and uncinate fasciculus [20].

In the literature, although many researches about brain localization of emotional prosody in mother tongue acquisition shows right hemisphere domination, recent findings, especially obtained from neuroimaging and electrophysiological studies do not entirely confirm a strong right hemisphere only. For example, in one neuroimaging study made with fMRI technique conducted on English native speakers, participants were asked to pay attention to visually presented words but had to make no overt response to them. As a result of the study, authors reported the increased activation in the left amygdala for both positive and negative emotional words [21]. Similarly, in another fMRI study carried out by Lewis et al. nineteen right-handed English native speakers were included, and it was discussed a dissociation of valence in subregions of the orbito-frontal cortex and arousal in the amygdala for visual word stimuli. In fact, left-lateralized amygdala activation in response to visually presented emotional word stimuli has been stated repeatedly [22]. Again, Mayer et al. examined the different prosodic states in German using with fMRI technique. In the study fabrication sound sequence was listened to the participants and it was interpreted in terms of focus, accent and emotional prosody. As a consequence of the study, it was observed the bilateral activation in left lower frontal gyrus, left insula and premotor cortex for focus condition. Authors also reported the anterior cingulate, supramarginal and posterior upper temporal gyrus activation for phonological process of emotional prosody system as well as syntactic process [23]. Likewise, Goerge et al. used a PET technique in their study which aimed to investigate the brain regions involved in understanding emotional prosody in mother tongue processing. As a result of the study, authors reported significant left prefrontal activation when participants were instructed to listen for content of verbal sentences compared with listening for prosody. In addition, authors also documented significantly more activation in the left than right frontal cortex [24].

However, it has observed in the literature that many of the electrophysiological studies about emotional prosody on mother tongue acquisition generally support the results of neuroimaging studies as well [25, 26, 27, 28]. For example, in one review study conducted by Kissler et al. scanned the Event-Related Brain Potential (ERP) researches. As a consequence of the study, authors reported enhanced Early Posterior Negativity (EPN) responses to emotional compared to neutral nouns during reading at left occipito-temporal electrode-sites [26]. In another ERP study, Herbert et al. demonstrated a bilaterally increased EPN for emotional prosody processing [28]. Likewise, in one comparative ERP study conducted by Pannekamp et al., emotional prosody process on English native speakers were evaluated, and authors stated that the prosody system can be seen not only in the right hemisphere of the brain but also in the left hemisphere [27]. The results of one electroencephalography (EEG) study findings obtained from Turkish native speakers also support the results of Pannekamp et al. [27] in a great extent. Similarly, in another EEG examination made by Arsic et al., it was showed the bilateral activation in both hemisphere for process of emotional prosody system as well [29].

Neither the aforementioned clinical studies nor neuroimaging and electrophysiological researches have established a clear picture of the neural substrates of emotional prosody in mother tongue acquisition. It is thought that varieties in methodology including different imaging techniques, different verbal stimuli and task instructions may be blame for these disparate findings.

3. Brain networks of emotional prosody in second language learning process

Although the mother tongue is a fast and effortless process, second language learning period includes some difficulties for many individuals [30]. Especially, “the expression and the perception of emotional states in a second language represent a difficult task for the learners. One of the reasons is the fact that, more than other aspects related to speech, the expression of emotional states in second language requires full control of the prosodic resources that contribute to their realization” [31].

Similar to studies investigated brain networks of emotional prosody on mother tongue, neuroimaging and neurophysiological studies about identifying brain localization of emotional prosody sequence in second language have produced controversial results in specifying resemblances versus differences as well. As a matter of the fact in one study conducted by Chen et al., it was investigated the neural mechanisms of emotional prosody processing in Chinese-English bilinguals by using both ERP and fMRI. According to ERP results, authors reported the reduced N400 for positive words compared to neutral words in second language. In fMRI, reduced activation was stated for mother tongue emotional words in both the left middle occipital gyrus and the left cerebellum whereas increased activation in the left cerebellum was reported for second language emotional words [32]. In another study, it was investigated the emotional prosody and verbal components of speech on English native speakers who learn Turkish as a second language using by fNIRS [1]. In the study four words (sınıf, alarm, doktor, gazete) were used for the experimental stimuli. The words were pronounced in angry, happy, neutral, and fearful tones. Among the four words, the phonetic pronunciations “alarm” and “doktor” have meaning in both Turkish and English. The other two words, ‘sınıf’ and ‘gazete’, although they were not a part of standard English lexicon but widely used in Turkish. As a consequence of the study, authors reported two novel and important results. First, they showed a strong right hemisphere laterization for emotional prosody in second language learning process, on the other hand, they demonstrated an increased activation in the primary and secondary cortex of the left hemisphere and the posterior superior temporal cortex due to the emotional intonation of speech stimuli. As a second, they remarked the significant increase activation in the right hemisphere for the perception of stimuli consisting only of Turkish words in English native speakers and do not know Turkish. However, authors reported a strong activation in the left hemisphere for all participants in stimuli consisting of words used mutual in both Turkish and English [1]. This study results support the fMRI research conducted by Buchanan et al. [10]. In the study, authors aimed to examine brain localization of emotional prosody on German and Turkish individuals who learn English as a second language. At the result of the study, authors indicated that the contrasts comparing language detection with emotion detection resulted in significantly lateralized activity in the frontal lobes, with increased right frontal activity during emotional detection and increased left frontal activity during verbal detection. In addition to this, specific analysis of the anterior auditory cortices of both hemispheres revealed greater right hemisphere activation during detection of emotion compared to activity associated with verbal detection [10]. In a similar way aforementioned studies about emotional prosody process in second language learning, in one fNIRS study Zhang et al. [14] used the database included language-like pseudosentences in Mandarin Chinese, which were constructed by replacing content words with semantically meaningless words. At the end of the study, authors demonstrated that several cerebral areas are critical for emotional prosody processing. Also, they confirmed that “the superior temporal cortex, especially the right middle and posterior parts of superior temporal gyrus, primarily works to discriminate between emotional and neutral prosodies”. Furthermore, according to the authors, the results suggested that categorization of emotions occurs within a high-level brain region the frontal cortex, since the brain activation patterns were distinct when positive (happy) were contrasted to negative (fearful and angry) prosody in the left middle part of inferior frontal gyrus and the frontal eye field and when angry were contrasted to neutral prosody in bilateral orbital frontal regions. These findings verified and extended previous fMRI findings in adult brain and also provided a developed version of brain activation for their following neonatal study [14].

Although the expression and the perception of emotional states in a second language process is notoriously difficult as opposed to mother tongue acquisition for learners, little is known about what changes are happening in the brain as decoding emotional prosody progress, when these changes occur during expression and perception of this sequence and how brain changes which reflect successes of learning can be determined. One of the reasons of this, in the literature, there are limited number of studies about brain networks of emotional prosody in second language learning. Also, these limited neuroimaging and neurophysiological studies have showed controversial results in specifying similarities versus differences foreign language learning neural networks. However; the knowledge of how the emotional prosody sequence works in the brain contribute to both mother tongue development and foreign language learning as well as clinical evaluation of individuals with verbal communication difficulty. Also, it is thought that determining brain networks of emotional prosody contributes to reorganize curricula for teaching emotional prosody in second language.

4. Conclusion

In this study, brain networks of emotional prosody in mother tongue acquisition and second language learning were tried to discuss according to the available results of electrophysiological, neuroimaging and clinical studies. Although different study results suggest activation of specific brain patterns is significantly correlated with process of emotional prosody, neither clinical studies nor neuroimaging and electrophysiological researches have established a clear picture of the neural substrates of emotional prosody in mother tongue acquisition and foreign language learning. Therefore, it is needed further experimental studies.