Xiaoming Jiang

By Xiaoming Jiang

This chapter provides a perspective from the latest EEG evidence in how brain signals enlighten the neurophysiological and neurocognitive mechanisms underlying the recognition of socioemotional expression conveyed in human speech and voice, drawing upon event‐related potentials’ studies (ERPs). Human sound can encode emotional meanings by different vocal parameters in words, real‐ vs. pseudo‐speeches, and vocalizations. Based on the ERP findings, recent development of the three‐stage model in vocal processing has highlighted initial‐ and late‐stage processing of vocal emotional stimuli. These processes, depending on which ERP components they were mapped onto, can be divided into the acoustic analysis, relevance and motivational processing, fine‐grained meaning analysis/integration/access, and higher‐level social inference, as the unfolding of the time scale. ERP studies on vocal socioemotions, such as happiness, anger, fear, sadness, neutral, sincerity, confidence, and sarcasm in the human voice and speech have employed different experimental paradigms such as crosssplicing, crossmodality priming, oddball, stroop, etc. Moreover, task demand and listener characteristics affect the neural responses underlying the decoding processes, revealing the role of attention deployment and interpersonal sensitivity in the neural decoding of vocal emotional stimuli. Cultural orientation affects our ability to decode emotional meaning in the voice. Neurophysiological patterns were compared between normal and abnormal emotional processing in the vocal expressions, especially in schizophrenia and in congenital amusia. Future directions highlight the study on human vocal expression aligning with other nonverbal cues, such as facial and body language, and the need to synchronize listener's brain potentials with other peripheral measures.

Part of the book: Emotion and Attention Recognition Based on Biological Signals and Images

By Xiaoming Jiang

Social context shapes our behavior in interpersonal communication. In this chapter, I will address how experimental psychology contributes to the study of socio-linguistic processes, focusing on nonverbal and verbal processing in a cross-cultural or cross-linguistic communicative setting. A systematic review of the most up-to-date empirical studies will show: 1) the culturally-universal and culturally-specific encoding of emotion in speech. The acoustic cues that are commonly involved in discriminating basic emotions in vocal expressions across languages and the cross-linguistic variations in such encoding will be demonstrated; 2) the modulation of in-group and out-group status (e.g. inferred from speaker’s dialect, familiarity towards a language) on the encoding and decoding of speaker’s meaning; 3) the impact of cultural orientation and cultural learning on the interpretation of social and affective meaning, focusing on how immigration process shapes one’s language use and comprehension. I will highlight the significance of combining the research paradigms from experimental psychology with cognitive (neuro)science methodologies such as electrophysiological recording and functional magnetic resonance imaging, to address the relevant questions in cross-cultural communicative settings. The chapter is concluded by a future direction to study the socio-cultural bases of language and linguistic underpinnings of cultural behaviour.

Part of the book: Sociolinguistics

By Xiaoming Jiang

One important question that remains open for the relationship between the brain and social behavior is whether and how prefrontal mechanisms responsible for social cognitive processes take place in language communication. Conventional studies have highlighted the role of inferior frontal gyrus (IFG) in processing context-independent linguistic information in speech and discourse. However, it is unclear how the medial prefrontal cortex (mPFC), the lateral prefrontal cortex (lPFC), and other structures (such as medial superior frontal gyrus, premotor cortex, anterior cingulate cortex, etc.) are involved when socially relevant language is encountered in real-life scenarios. Emerging neuroimaging and patient studies have suggested the association of prefrontal regions with individual differences and impairments in the comprehension of speech act, nonliteral language, or construction-based pragmatic information. By summarizing and synthesizing the most recent functional magnetic resonance imaging (fMRI) studies, this chapter aims to show how neurocognitive components underlying the social function of prefrontal cortex support pragmatic language processing, such as weighing relevant social signals, resolving ambiguities, and identifying hidden speaker meanings. The conclusion lends impact on an emerging interest in neuropragmatics and points out a promising line of research to address the mediating role of prefrontal cortex in the relation of language and social cognition.

Part of the book: Prefrontal Cortex

By Xiaoming Jiang

This chapter provides bibliometric analyses of novel trends in the research toward pragmatic aspects of language processing and learning in the studies of psycholinguistics and second-language acquisition. Growing interests in the relevant themes are shown with the analysis of the co-occurrence of keywords in a common literature and the bibliographic coupling between literatures. The emergence of novel experimental methodologies, including the application of neuroimaging and machine learning approaches to the psycholinguistic research, provides new opportunities of looking into the pragmatic aspects of language acquisition and invites new empirical research to validate the theories and extend the boundaries of second-language acquisition research in the real-world setting.

Part of the book: Second Language Acquisition

By Xiaoming Jiang

Communicative expression is a cross-species phenomenon. We investigated the perceptual attributes of social expressions encoded in human-like animal stickers commonly used as nonverbal communicative tools on social media (e.g. WeChat). One hundred and twenty animal stickers which varied in 12 categories of social expressions (serving pragmatic or emotional functions), 5 animal kinds (cats, dogs, ducks, rabbits, pigs) and 2 presented forms (real animal vs. cartoon animal) were presented to social media users, who were asked to rate on the human likeness, the cuteness, the expressiveness and the matchness of each intended expression against the given label. The data shows that the kind of animal that is expected to best encode a certain expression is modulated by its presented forms. The “cuteness” stereotype towards a certain kind of animal is sometimes violated as a function of the presented forms. Moreover, user’s gender, interpersonal sensitivity and attitudes towards the ethic use of animals modulated various perceptual attributes. These findings highlight the factors underlying the decoding of social meanings in human-like animal stickers as nonverbal cues in virtual communication.

Part of the book: Types of Nonverbal Communication

By Xiaoming Jiang

Part of the book: Types of Nonverbal Communication

By Qi Yang, Xiaohua Cao and Xiaoming Jiang

Becoming multilingual has a broad impact on cognitive abilities, especially visual processing. An important theoretical issue is whether the acquisition of distinct script systems affects face processing in an identical way, or, if not, how this acquisition may exert differential impacts on face processing. By reviewing the existing literature, we propose that Asian participants with the logographic script system differ from Western counterparts with the alphabetic script system in viewing faces. The contribution of the chapter is to identify the possible role of types of script systems in face processing mechanisms and to put forward the research direction in the future with several new methodological efforts.

Part of the book: Multilingualism

By Xiaoming Jiang and Chenkai Lin

Part of the book: Interdisciplinary Insights on Interpersonal Relationships