Abstract
Cross-modality in function is a fundamental ability in humans and is closely associated with the basic functions. Several studies have demonstrated that vision strongly influences other senses such as hearing, touch, taste, and smell. However, the dysfunction in this cross-modality caused by disease, is poorly understood. In addition to evidence that Parkinson’s disease (PD) impairs various cognitive functions including olfaction, a recent study showed that olfactory function is unaffected by visual information in patients with PD. This finding suggests that the link between vision and olfaction is underactive in PD. This chapter reviews the cross-modal dysfunction and dwells on the possibility of a novel precursor assessment for PD.
Keywords
- cross-modality
- vision
- olfaction
- Parkinson’s disease
- striatum
1. Introduction
Sensory organs independently process the corresponding external stimuli such as light, sound, pressure, taste, and smell. However, one sense rarely acts alone during the perception/cognition of a given event in daily life. Our experiences are dependent on the integration of the visual, auditory, somesthetic, gustatory, and olfactory systems. For example, during the act of smelling a strawberry, the sense of smell may vary according to the color of the strawberry (whether it is bright red or green-tinged), even if the physical odorant is the same (Figure 1). This example demonstrates that cognition is built upon cross-modality of function. The regions associated with cross-modality are spread throughout a wide area in brain [1]. For example, the integration of vision and touch is mainly associated with the right posterior fusiform gyrus and primary somatosensory cortices [2], whereas the integration of vision and audition is mainly associated with the right frontal lobe and right superior temporal gyrus [3].

Figure 1.
An example of the effect of color effect on smell recognition. The red strawberry on the left may seem to have a more pleasant odor than the green strawberry on the right.
The accuracy of each of the senses is different, and vision is often prioritized over other senses during integration. Several studies have demonstrated that visual input strongly influences hearing [4] touch [5], taste [6], and smell [7, 8] (Figure 2); however, the reverse, in which other senses influence vision, is a rare phenomenon, excluding the auditory-vision relationships such as the McGurk effect [9] or double flash illusion [10]. For the McGurk effect, on a video of audiovisual speech, if a lip movements show a “ba-ba” sound, whereas an auditory information is that of “ga-ga”, most people experiences an illusory sound “da-da”. For the double flash illusion, if one dot flashes on the display when two beeps are sounded, most people reports experiencing two flashes. Thus, vision dominates the other senses in many cases. Although the mechanism of cross-modality has become increasingly clear in healthy persons [1, 2, 3, 4, 5, 6, 7, 8, 9, 10], a function in disease states remains unclear.

Figure 2.
Diagram representing the effective relationship among the five senses. Vision is often prioritized over the other senses during integration.
2. Clinical state and cognitive dysfunction in Parkinson’s disease
Parkinson’s disease (PD) causes tremors of the hands, stiffness, akinesia, or inability to maintain posture. Patients with PD have decreased levels of dopamine in the striatum, which consists of the putamen and caudate nucleus, which are a part of the basal ganglia [11]. They may further develop protein-related disorders, such as the presynaptic dopamine transporter (DaT), which is responsible for the incorporation and transmission of dopamine components [12]. DaT scanning is performed with ioflupane (123I-FP-CIT), a radio-iodinated cocaine analogue [13, 14]. It has a high affinity for the DaT protein located on presynaptic nerve endings in the striatum. These nerve endings are projections of dopaminergic neurons from the substantia nigra. Binding of a radiopharmaceutical agent to DaT reflects number of striatal dopaminergic neurons. The accumulation of DaT is expressed in proportion to the occipital lobe (Figure 3). The degree of DaT deficit is associated with the severity of the movement disorder [15].

Figure 3.
Striatal DaT deficit in Parkinson’s disease (coronal view). The left panel shows a binding radiopharmaceutical agent accumulation in a healthy person. The right panel shows a binding radiopharmaceutical agent accumulation in a patient with Parkinson’s disease. The numbers indicate binding radiopharmaceutical agent counts on the striatum per pixel.
The principal symptoms of PD are related to movement, although the non-motor symptoms are also noteworthy. For example, attention function, which demands a response speed or switching [16]; executive function, which is related to action planning or problem solving [17]; working memory function, which holds information temporarily and allocates attentional resources [18, 19]; social cognition function, which involves interpreting emotions based on others’ facial expressions [20, 21]; and temporal function, which estimates duration [22, 23] were shown to be impaired in PD.
Impairment of olfaction has also been reported in patients with PD and may be a biomarker for cognitive dysfunction and early PD [24, 25, 26]. Olfactory information is projected directly to the limbic system, including the piriform (PIR), amygdala (AMG), hippocampus (HI), and entorhinal cortex (ENT). These areas determine odor detection, its emotional evaluation (pleasant or unpleasant), and memory retrieval [27]. Olfactory information finally ascends to the orbitofrontal cortex (OFC). The OFC participates in the identification or recognition of odor, filtered through emotion and memory via activation of the AMG and HI [28]. Olfactory dysfunction in PD may occur due to deficiency of dopamine and pathological changes in the ENT, AMG and HI, especially in the areas affected by early onset of PD [29].
Furthermore, the striatum is involved in various functions, which include an integration of sensory information [30, 31, 32]. Studies have demonstrated that the striatum (putamen and caudate) acts as a “hub,” with specialized functional roles for different neuron types in mice [33] and humans [34, 35]. However, it was unclear whether PD affects cross-modal function.
3. Cross-modality dysfunction in Parkinson’s disease
Recently, a study showed that PD causes a decline in the cross-modal function of vision and olfaction [36]. This study conducted behavioral experiments to identify the influence of PD on cross-modal function by comparing the behavior of patients with PD with that of healthy controls. The principal aim of this study was to measure odor-strength perception and preference, while presenting smells paired with visual information, to characterize vision/olfaction integration in patients with PD.
In the experiment, odor detection thresholds in each participant were first determined using an olfactometer, which has five odorants (β-phenylethyl alcohol, methyl cyclopentenolone, isovaleric acid, γ-undecalactone, and skatole). For example, methyl cyclopentenolone smells like caramel pudding (pleasant) and skatole smells like rotten vegetables (unpleasant). The study employed detectable odorant thresholds in each of five categories and prepared five original pictures associated with the five odorants of each category. For example, the category “caramel pudding” consisted of a picture of “pudding” and the odorant “methyl cyclopentenolone,” whereas “rotten vegetables” consisted of “rotten vegetables” and the odorant “skatole” (Figure 4). The “control” category consisted of a noise picture and an odorless liquid. Four combinations were arranged: the original picture with the original odorant (combination “A”), the control picture with the original odorant (combination “B”), and the original picture with the control odorant (combination “C”). A control combination was added: the control picture with the control odorant (combination “D”) (Figure 5). Participants were asked take a sniff while viewing the picture, and were subsequently asked to evaluate the strength (weak – strong) and preference (pleasant – unpleasant) of each odor on a visual analog scale (VAS).

Figure 4.
Examples of the category type. Each category consists of an original odorant and an original picture corresponding to the odorant, and a control category consisting of a noise picture and an odorless liquid (this figure is cited with edit from a part of Honma et al. [

Figure 5.
Combinations of the pictures and odorants. Combination a includes the original picture and original odorant. Combination B includes the control picture and original odorant. Combination C includes the original picture and the control odorant. Combination D includes the control picture and control odorant (this figure is cited with edit from a part of Honma et al. [
In the

Figure 6.
(A) Strength of the odor represented on the visual analog scale. (B) Preference for odor on the visual analog scale. The visual analog scale scores of the groups (healthy controls and participants with PD) and combinations (A, B, and C) were compared for each category. The control category (combination D) was analyzed independently. Asterisks indicate significant differences. These results show that the visual input affects odor estimation in healthy controls, with little effect in PD (this figure is cited with edit from a part of Honma et al. [
Furthermore, the study reported a possible effect of striatal DaT deficits in patients with PD on the olfaction-vision cross-modality. DaT imaging indicated that striatal DaT deficit in PD, especially that in the posterior putamen, is associated with the cross-modal effect of perception on odor preference, and the laterality may depend on the emotional category (pleasant or unpleasant) (Figure 7).

Figure 7.
Regression analysis identified the left posterior putamen as the independent variable for the cross-modal effects of pleasant categories (caramel pudding, fresh roses, and canned peaches). The right posterior putamen was identified as the independent variable for the cross-modal effects of unpleasant categories (rotten vegetables and dirty socks). The laterality in putamen may depend on emotionality.
4. Possible mechanism of dysfunctional cross-modality between vision and olfaction
Honma et al. [36] showed that the olfactory function was unaffected by visual information in patients with PD, supporting the hypothesis that PD impairs cross-modality between vision and olfaction [36]. Healthy participants tend to overestimate odor when presented with an original picture, for example, a picture of
Olfactory perception is ambiguous, and a smell may be hard to identify without verbal or visual assistance. It is true that olfaction is modulated by visual elements [7, 8]. This multimodal integration is responsible for the OFC, which receives information from visual association, and the olfactory, gustatory, somatosensory and, auditory areas [37]. Recent brain imaging studies showed that the OFC participates in vision-olfaction integration in healthy individuals [7]. Gottfried and Dolan (2003) noted higher activation of the OFC when a smell is presented with a word label [7]. Cognitive factors, such as visual stimuli modulate representations of odor at a relatively early level of cortical processing, known as the top-down cognitive influence, which directly affects emotion [38]. Healthy participants exhibited dominance of the visual sense in this study. On the other hand, patients with PD exhibited decrease dominance of visual information. They concentrated more on olfactory perception, without modulation from visual input. The OFC in patients with PD is known to be less active for all modalities during stimulation, including olfaction [39]. Decreased activation of the OFC in patients with PD may partly account for declining cross-modality. Moreover, detection and cognition levels for odor were lower than those in the controls. During olfaction, the OFC also integrates information from the AMG and HI, which play a role in emotional evaluation and memory retrieval [37]. Reduction of OFC function may lead to deficits in recognition and identification of odor. Thus, patients with PD may tend to focus more on smell detection, which may need activation of the basic primary olfactory areas, such as the ENT and PIR.
The relationship of DaT levels in PD with odor preferences demonstrated by the study is significant. However, it has been reported that administration of dopamine agonists (levodopa) does not influence the olfactory deficit. Thus, dopamine loss may not affect olfaction [40]. Here, it was not possible to establish a direct link between olfaction and dopamine levels, as measured by DaT in the putamen. However, earlier findings suggest that the putamen may play a role in sensory integration [34, 35]. Lack of dopamine linked to deficient DaT protein in the striatum leads to a decline in dopamine levels in several regions [41]. The corticostriatal loop sends signals that pass through brain regions, such as the striatum–pallidus–thalamic–cortex [42]. Dopamine regulates the prefrontal-AMG circuit, which plays an important role in emotion processing [43]. This suggests that vision-olfaction integration may be influenced by dopamine signals, via a striatum-centered network. Dopamine deficiency in PD may affect vision-olfaction integration, including emotion and cognitive processing.
The laterality of DaT level in the putamen related to odor preference is of further interest. That is, the left is associated with pleasant smells and the right is associated with unpleasant smells. A recent study has shown that a pleasant odor is associated with bilateral or left AMG activation, and an unpleasant odor is associated with activation of the right AMG [27]. The left/right difference for smell-evoked emotion may be linked to AMG processing, because these regions are strongly connected at a fiber level [44].
5. Conclusion
It is essential to investigate the onset of cross-modality dysfunction in the future. Studies are currently focusing on early detection of PD, including signs of declining olfactory ability and rapid eye movement-related sleep disorders, as precursory biomarkers for PD [24, 25, 26, 45]. This approach may provide a new view of precursors, if the dysfunction develops before onset of movement disorders in PD (Figure 8). Furthermore, it is necessary to examine whether cross-modal dysfunction occurs in other diseases with striatum deficit, such as multiple system atrophy [46] and Huntington’s disease [47]. Cross-modal dysfunction may also lead to diagnoses of other diseases.

Figure 8.
Conceivable onsets of cross-modality dysfunction. Declining olfactory ability and rapid eye movement-related sleep disorder are known to be precursory biomarkers for Parkinson’s disease. Cross-modality dysfunction also has the potential of becoming a novel biomarker.
Acknowledgments
The author thanks his original collaborators, Dr. Masaoka, Dr. Kuroda, Dr. Futamura, Dr. Shiromaru, Dr. Izumizaki, and Dr. Kawamura (Showa University School of Medicine). This paper was supported by Grant-in-aids for Scientific Research (C) (No. 18 K03185).
References
- 1.
Sperdin HF, Cappe C, Murray MM. The behavioral relevance of multisensory neural response interactions. Frontiers in Neuroscience. 2010; 4 :9. DOI: 10.3389/neuro.01.009.2010 - 2.
Helbig HB, Ernst MO, Ricciardi E, Pietrini P, Thielscher A, Mayer KM, et al. The neural mechanisms of reliability weighted integration of shape information from vision and touch. NeuroImage. 2012; 60 :1063-1072. DOI: 10.1016/j.neuroimage.2011.09.072 - 3.
Kawashima R, Watanabe J, Kato T, Nakamura A, Hatano K, Schormann T, et al. Direction of cross-modal information transfer affects human brain activation: A PET study. The European Journal of Neuroscience. 2002; 16 :137-144 - 4.
Bertelson P, Radeau M. Cross-modal bias and perceptual fusion with auditory-visual spatial discordance. Perception & Psychophysics. 1981; 29 :578-584 - 5.
Pavani F, Spence C, Driver J. Visual capture of touch: Out-of-the-body experiences with rubber gloves. Psychological Science. 2000; 11 :353-359 - 6.
Maga JA. Influence of color on taste thresholds. Chemical Senses. 1974; 1 :115-119 - 7.
Gottfried JA, Dolan RJ. The nose smells what the eye sees: Crossmodal visual facilitation of human olfactory perception. Neuron. 2003; 39 :375-386 - 8.
Luisa Demattè M, Sanabria D, Spence C. Cross-modal associations between odors and colors. Chemical Senses. 2006; 31 :531-538 - 9.
McGurk H, MacDonald J. Hearing lips and seeing voices. Nature. 1976; 264 :746-748 - 10.
Shams L, Kamitani Y, Shimojo S. Visual illusion induced by sound. Brain Research. Cognitive Brain Research. 2002; 14 :147-152 - 11.
Haber SN. The place of dopamine in the cortico-basal ganglia circuit. Neuroscience. 2014; 282 :248-257. DOI: 10.1016/j.neuroscience.2014.10.008 - 12.
Vaughan RA, Foster JD. Mechanisms of dopamine transporter regulation in normal and disease states. Trends in Pharmacological Sciences. 2013; 34 :489-496. DOI: 10.1016/j.tips.2013.07.005 - 13.
Kägi G, Bhatia KP, Tolosa E. The role of DAT-SPECT in movement disorders. Journal of Neurology, Neurosurgery, and Psychiatry. 2010; 81 :5-12. DOI: 10.1136/jnnp.2008.157370 - 14.
Tatsch K, Poepperl G. Nigrostriatal dopamine terminal imaging with dopamine transporter SPECT: An update. Journal of Nuclear Medicine. 2013; 54 :1331-1338. DOI: 10.2967/jnumed.112.105379 - 15.
Jaakkola E, Joutsa J, Kaasinen V. Predictors of normal and abnormal outcome in clinical brain dopamine transporter imaging. Journal of Neural Transmission. 2016; 123 :205-209. DOI: 10.1007/s00702-015-1495-0 - 16.
Cerasa A, Gioia MC, Salsone M, Donzuso G, Chiriaco C, Realmuto S, et al. Neurofunctional correlates of attention rehabilitation in Parkinson’s disease: An explorative study. Neurological Sciences. 2014; 35 :1173-1180. DOI: 10.1007/s10072-014-1666-z - 17.
Trujillo JP, Gerrits NJ, Vriend C, Berendse HW, van den Heuvel OA, van der Werf YD. Impaired planning in Parkinson’s disease is reflected by reduced brain activation and connectivity. Human Brain Mapping. 2015; 36 :3703-3715. DOI: 10.1002/hbm.22873 - 18.
Ventre-Dominey J, Bourret S, Mollion H, Broussolle E, Dominey PF. Dissociable dorsal and ventral frontostriatal working memory circuits: Evidence from subthalamic stimulation in Parkinson’s disease. Human Brain Mapping. 2014; 35 :552-566. DOI: 10.1002/hbm.22205 - 19.
Moustafa AA, Bell P, Eissa AM, Hewedi DH. The effects of clinical motor variables and medication dosage on working memory in Parkinson’s disease. Brain and Cognition. 2013; 82 :137-145. DOI: 10.1016/j.bandc.2013.04.001 - 20.
Alonso-Recio L, Serrano JM, Martín P. Selective attention and facial expression recognition in patients with Parkinson’s disease. Archives of Clinical Neuropsychology. 2014; 29 :374-384. DOI: 10.1093/arclin/acu018 - 21.
Sprengelmeyer R, Young AW, Mahn K, Schroeder U, Woitalla D, Büttner T, et al. Facial expression recognition in people with medicated and unmedicated Parkinson’s disease. Neuropsychologia. 2003; 41 :1047-1057 - 22.
Honma M, Kuroda T, Futamura A, Shiromaru A, Kawamura M. Dysfunctional counting of mental time in Parkinson’s disease. Scientific Reports. 2016; 6 :25421. DOI: 10.1038/srep25421 - 23.
Honma M, Murai Y, Shima S, Yotsumoto Y, Kuroda T, Futamura A, et al. Spatial distortion related to time compression during spatiotemporal production in Parkinson’s disease. Neuropsychologia. 2017; 102 :61-69. DOI: 10.1016/j.neuropsychologia.2017.06.004 - 24.
Doty RL, Deems DA, Stellar S. Olfactory dysfunction in parkinsonism: A general deficit unrelated to neurologic signs, disease stage, or disease duration. Neurology. 1988; 38 :1237-1244 - 25.
Masaoka Y, Yoshimura N, Inoue M, Kawamura M, Homma I. Impairment of odor recognition in Parkinson’s disease caused by weak activations of the orbitofrontal cortex. Neuroscience Letters. 2007; 412 :45-50 - 26.
Mahlknecht P, Pechlaner R, Boesveldt S, Volc D, Pinter B, Reiter E, et al. Optimizing odor identification testing as quick and accurate diagnostic tool for Parkinson’s disease. Movement Disorders. 2016; 31 :1408-1413. DOI: 10.1002/mds.26637 - 27.
Patin A, Pause BM. Human amygdala activations during nasal chemoreception. Neuropsychologia. 2015; 78 :171-194. DOI: 10.1016/j.neuropsychologia.2015.10.009 - 28.
Rolls ET. The rules of formation of the olfactory representations found in the orbitofrontal cortex olfactory areas in primates. Chemical Senses. 2001; 26 :595-604 - 29.
Braak H, Del Tredici K. Neuroanatomy and pathology of sporadic Parkinson’s disease. Advances in Anatomy, Embryology, and Cell Biology. 2009; 201 :1-119 - 30.
Chudler EH, Dong WK. The role of the basal ganglia in nociception and pain. Pain. 1995; 60 :3-38 - 31.
Schultz W. Reward functions of the basal ganglia. Journal of Neural Transmission. 2016; 123 :679-693. DOI: 10.1007/s00702-016-1510-0 - 32.
Middleton FA, Strick PL. Basal ganglia and cerebellar loops: Motor and cognitive circuits. Brain Research. Brain Research Reviews. 2000; 31 :236-250 - 33.
Reig R, Silberberg G. Multisensory integration in the mouse striatum. Neuron. 2014; 83 :1200-1212. DOI: 10.1016/j.neuron.2014.07.033 - 34.
Gentile G, Petkova VI, Ehrsson HH. Integration of visual and tactile signals from the hand in the human brain: An FMRI study. Journal of Neurophysiology. 2011; 105 :910-922. DOI: 10.1152/jn.00840.2010 - 35.
von Saldern S, Noppeney U. Sensory and striatal areas integrate auditory and visual signals into behavioral benefits during motion discrimination. Journal of Neuroscience. 2013; 33 :8841-8849. DOI: 10.1523/JNEUROSCI.3020-12.2013 - 36.
Honma M, Masaoka Y, Kuroda T, Futamura A, Shiromaru A, Izumizaki M, et al. Impairment of cross-modality of vision and olfaction in Parkinson disease. Neurology. 2018; 90 :e977-e984. DOI: 10.1212/WNL.0000000000005110 - 37.
Rolls ET, Baylis LL. Gustatory, olfactory, and visual convergence within the primate orbitofrontal cortex. Journal of Neuroscience. 1994; 14 :5437-5452 - 38.
Rolls ET. Top-down control of visual perception: Attention in natural vision. Perception. 2008; 37 :333-354 - 39.
Masaoka Y, Pantelis C, Phillips A, Kawamura M, Mimura M, Minegishi G, et al. Markers of brain illness may be hidden in your olfactory ability: A Japanese perspective. Neuroscience Letters. 2013; 549 :182-185. DOI: 10.1016/j.neulet.2013.05.077 - 40.
Doty RL, Stern MB, Pfeiffer C, Gollomp SM, Hurtig HI. Bilateral olfactory dysfunction in early stage treated and untreated idiopathic Parkinson’s disease. Journal of Neurology, Neurosurgery, and Psychiatry. 1992; 55 :138-142 - 41.
DeLong MR. Primate models of movement disorders of basal ganglia origin. Trends in Neurosciences. 1990; 13 :281-285 - 42.
Galvan A, Devergnas A, Wichmann T. Alterations in neuronal activity in basal ganglia-thalamocortical circuits in the parkinsonian state. Frontiers in Neuroanatomy. 2015; 9 :5. DOI: 10.3389/fnana.2015.00005 - 43.
Rey CD, Lipps J, Shansky RM. Dopamine D1 receptor activation rescues extinction impairments in low-estrogen female rats and induces cortical layer-specific activation changes in prefrontal-amygdala circuits. Neuropsychopharmacology. 2014; 39 :1282-1289. DOI: 10.1038/npp.2013.338 - 44.
Gattass R, Galkin TW, Desimone R, Ungerleider LG. Subcortical connections of area V4 in the macaque. The Journal of Comparative Neurology. 2014; 522 :1941-1965. DOI: 10.1002/cne.23513 - 45.
Chen MC, Yu H, Huang ZL, Lu J. Rapid eye movement sleep behavior disorder. Current Opinion in Neurobiology. 2013; 23 :793-798. DOI: 10.1016/j.conb.2013.02.019 - 46.
Cilia R, Marotta G, Benti R, Pezzoli G, Antonini A. Brain SPECT imaging in multiple system atrophy. Journal of Neural Transmission. 2005; 112 :1635-1645 - 47.
Rüb U, Seidel K, Heinsen H, Vonsattel JP, den Dunnen WF, Korf HW. Huntington’s disease (HD): The neuropathology of a multisystem neurodegenerative disorder of the human brain. Brain Pathology. 2016; 26 :726-740. DOI: 10.1111/bpa.12426