Cross-modality in function is a fundamental ability in humans and is closely associated with the basic functions. Several studies have demonstrated that vision strongly influences other senses such as hearing, touch, taste, and smell. However, the dysfunction in this cross-modality caused by disease, is poorly understood. In addition to evidence that Parkinson’s disease (PD) impairs various cognitive functions including olfaction, a recent study showed that olfactory function is unaffected by visual information in patients with PD. This finding suggests that the link between vision and olfaction is underactive in PD. This chapter reviews the cross-modal dysfunction and dwells on the possibility of a novel precursor assessment for PD.
- Parkinson’s disease
Sensory organs independently process the corresponding external stimuli such as light, sound, pressure, taste, and smell. However, one sense rarely acts alone during the perception/cognition of a given event in daily life. Our experiences are dependent on the integration of the visual, auditory, somesthetic, gustatory, and olfactory systems. For example, during the act of smelling a strawberry, the sense of smell may vary according to the color of the strawberry (whether it is bright red or green-tinged), even if the physical odorant is the same (Figure 1). This example demonstrates that cognition is built upon cross-modality of function. The regions associated with cross-modality are spread throughout a wide area in brain . For example, the integration of vision and touch is mainly associated with the right posterior fusiform gyrus and primary somatosensory cortices , whereas the integration of vision and audition is mainly associated with the right frontal lobe and right superior temporal gyrus .
The accuracy of each of the senses is different, and vision is often prioritized over other senses during integration. Several studies have demonstrated that visual input strongly influences hearing  touch , taste , and smell [7, 8] (Figure 2); however, the reverse, in which other senses influence vision, is a rare phenomenon, excluding the auditory-vision relationships such as the McGurk effect  or double flash illusion . For the McGurk effect, on a video of audiovisual speech, if a lip movements show a “ba-ba” sound, whereas an auditory information is that of “ga-ga”, most people experiences an illusory sound “da-da”. For the double flash illusion, if one dot flashes on the display when two beeps are sounded, most people reports experiencing two flashes. Thus, vision dominates the other senses in many cases. Although the mechanism of cross-modality has become increasingly clear in healthy persons [1, 2, 3, 4, 5, 6, 7, 8, 9, 10], a function in disease states remains unclear.
2. Clinical state and cognitive dysfunction in Parkinson’s disease
Parkinson’s disease (PD) causes tremors of the hands, stiffness, akinesia, or inability to maintain posture. Patients with PD have decreased levels of dopamine in the striatum, which consists of the putamen and caudate nucleus, which are a part of the basal ganglia . They may further develop protein-related disorders, such as the presynaptic dopamine transporter (DaT), which is responsible for the incorporation and transmission of dopamine components . DaT scanning is performed with ioflupane (123I-FP-CIT), a radio-iodinated cocaine analogue [13, 14]. It has a high affinity for the DaT protein located on presynaptic nerve endings in the striatum. These nerve endings are projections of dopaminergic neurons from the substantia nigra. Binding of a radiopharmaceutical agent to DaT reflects number of striatal dopaminergic neurons. The accumulation of DaT is expressed in proportion to the occipital lobe (Figure 3). The degree of DaT deficit is associated with the severity of the movement disorder .
The principal symptoms of PD are related to movement, although the non-motor symptoms are also noteworthy. For example, attention function, which demands a response speed or switching ; executive function, which is related to action planning or problem solving ; working memory function, which holds information temporarily and allocates attentional resources [18, 19]; social cognition function, which involves interpreting emotions based on others’ facial expressions [20, 21]; and temporal function, which estimates duration [22, 23] were shown to be impaired in PD.
Impairment of olfaction has also been reported in patients with PD and may be a biomarker for cognitive dysfunction and early PD [24, 25, 26]. Olfactory information is projected directly to the limbic system, including the piriform (PIR), amygdala (AMG), hippocampus (HI), and entorhinal cortex (ENT). These areas determine odor detection, its emotional evaluation (pleasant or unpleasant), and memory retrieval . Olfactory information finally ascends to the orbitofrontal cortex (OFC). The OFC participates in the identification or recognition of odor, filtered through emotion and memory via activation of the AMG and HI . Olfactory dysfunction in PD may occur due to deficiency of dopamine and pathological changes in the ENT, AMG and HI, especially in the areas affected by early onset of PD .
Furthermore, the striatum is involved in various functions, which include an integration of sensory information [30, 31, 32]. Studies have demonstrated that the striatum (putamen and caudate) acts as a “hub,” with specialized functional roles for different neuron types in mice  and humans [34, 35]. However, it was unclear whether PD affects cross-modal function.
3. Cross-modality dysfunction in Parkinson’s disease
Recently, a study showed that PD causes a decline in the cross-modal function of vision and olfaction . This study conducted behavioral experiments to identify the influence of PD on cross-modal function by comparing the behavior of patients with PD with that of healthy controls. The principal aim of this study was to measure odor-strength perception and preference, while presenting smells paired with visual information, to characterize vision/olfaction integration in patients with PD.
In the experiment, odor detection thresholds in each participant were first determined using an olfactometer, which has five odorants (β-phenylethyl alcohol, methyl cyclopentenolone, isovaleric acid, γ-undecalactone, and skatole). For example, methyl cyclopentenolone smells like caramel pudding (pleasant) and skatole smells like rotten vegetables (unpleasant). The study employed detectable odorant thresholds in each of five categories and prepared five original pictures associated with the five odorants of each category. For example, the category “caramel pudding” consisted of a picture of “pudding” and the odorant “methyl cyclopentenolone,” whereas “rotten vegetables” consisted of “rotten vegetables” and the odorant “skatole” (Figure 4). The “control” category consisted of a noise picture and an odorless liquid. Four combinations were arranged: the original picture with the original odorant (combination “A”), the control picture with the original odorant (combination “B”), and the original picture with the control odorant (combination “C”). A control combination was added: the control picture with the control odorant (combination “D”) (Figure 5). Participants were asked take a sniff while viewing the picture, and were subsequently asked to evaluate the strength (weak – strong) and preference (pleasant – unpleasant) of each odor on a visual analog scale (VAS).
Furthermore, the study reported a possible effect of striatal DaT deficits in patients with PD on the olfaction-vision cross-modality. DaT imaging indicated that striatal DaT deficit in PD, especially that in the posterior putamen, is associated with the cross-modal effect of perception on odor preference, and the laterality may depend on the emotional category (pleasant or unpleasant) (Figure 7).
4. Possible mechanism of dysfunctional cross-modality between vision and olfaction
Honma et al.  showed that the olfactory function was unaffected by visual information in patients with PD, supporting the hypothesis that PD impairs cross-modality between vision and olfaction . Healthy participants tend to overestimate odor when presented with an original picture, for example, a picture of
Olfactory perception is ambiguous, and a smell may be hard to identify without verbal or visual assistance. It is true that olfaction is modulated by visual elements [7, 8]. This multimodal integration is responsible for the OFC, which receives information from visual association, and the olfactory, gustatory, somatosensory and, auditory areas . Recent brain imaging studies showed that the OFC participates in vision-olfaction integration in healthy individuals . Gottfried and Dolan (2003) noted higher activation of the OFC when a smell is presented with a word label . Cognitive factors, such as visual stimuli modulate representations of odor at a relatively early level of cortical processing, known as the top-down cognitive influence, which directly affects emotion . Healthy participants exhibited dominance of the visual sense in this study. On the other hand, patients with PD exhibited decrease dominance of visual information. They concentrated more on olfactory perception, without modulation from visual input. The OFC in patients with PD is known to be less active for all modalities during stimulation, including olfaction . Decreased activation of the OFC in patients with PD may partly account for declining cross-modality. Moreover, detection and cognition levels for odor were lower than those in the controls. During olfaction, the OFC also integrates information from the AMG and HI, which play a role in emotional evaluation and memory retrieval . Reduction of OFC function may lead to deficits in recognition and identification of odor. Thus, patients with PD may tend to focus more on smell detection, which may need activation of the basic primary olfactory areas, such as the ENT and PIR.
The relationship of DaT levels in PD with odor preferences demonstrated by the study is significant. However, it has been reported that administration of dopamine agonists (levodopa) does not influence the olfactory deficit. Thus, dopamine loss may not affect olfaction . Here, it was not possible to establish a direct link between olfaction and dopamine levels, as measured by DaT in the putamen. However, earlier findings suggest that the putamen may play a role in sensory integration [34, 35]. Lack of dopamine linked to deficient DaT protein in the striatum leads to a decline in dopamine levels in several regions . The corticostriatal loop sends signals that pass through brain regions, such as the striatum–pallidus–thalamic–cortex . Dopamine regulates the prefrontal-AMG circuit, which plays an important role in emotion processing . This suggests that vision-olfaction integration may be influenced by dopamine signals, via a striatum-centered network. Dopamine deficiency in PD may affect vision-olfaction integration, including emotion and cognitive processing.
The laterality of DaT level in the putamen related to odor preference is of further interest. That is, the left is associated with pleasant smells and the right is associated with unpleasant smells. A recent study has shown that a pleasant odor is associated with bilateral or left AMG activation, and an unpleasant odor is associated with activation of the right AMG . The left/right difference for smell-evoked emotion may be linked to AMG processing, because these regions are strongly connected at a fiber level .
It is essential to investigate the onset of cross-modality dysfunction in the future. Studies are currently focusing on early detection of PD, including signs of declining olfactory ability and rapid eye movement-related sleep disorders, as precursory biomarkers for PD [24, 25, 26, 45]. This approach may provide a new view of precursors, if the dysfunction develops before onset of movement disorders in PD (Figure 8). Furthermore, it is necessary to examine whether cross-modal dysfunction occurs in other diseases with striatum deficit, such as multiple system atrophy  and Huntington’s disease . Cross-modal dysfunction may also lead to diagnoses of other diseases.
The author thanks his original collaborators, Dr. Masaoka, Dr. Kuroda, Dr. Futamura, Dr. Shiromaru, Dr. Izumizaki, and Dr. Kawamura (Showa University School of Medicine). This paper was supported by Grant-in-aids for Scientific Research (C) (No. 18 K03185).
Conflict of interest
The author of this manuscript has no conflict of interest.
Sperdin HF, Cappe C, Murray MM. The behavioral relevance of multisensory neural response interactions. Frontiers in Neuroscience. 2010; 4:9. DOI: 10.3389/neuro.01.009.2010
Helbig HB, Ernst MO, Ricciardi E, Pietrini P, Thielscher A, Mayer KM, et al. The neural mechanisms of reliability weighted integration of shape information from vision and touch. NeuroImage. 2012; 60:1063-1072. DOI: 10.1016/j.neuroimage.2011.09.072
Kawashima R, Watanabe J, Kato T, Nakamura A, Hatano K, Schormann T, et al. Direction of cross-modal information transfer affects human brain activation: A PET study. The European Journal of Neuroscience. 2002; 16:137-144
Bertelson P, Radeau M. Cross-modal bias and perceptual fusion with auditory-visual spatial discordance. Perception & Psychophysics. 1981; 29:578-584
Pavani F, Spence C, Driver J. Visual capture of touch: Out-of-the-body experiences with rubber gloves. Psychological Science. 2000; 11:353-359
Maga JA. Influence of color on taste thresholds. Chemical Senses. 1974; 1:115-119
Gottfried JA, Dolan RJ. The nose smells what the eye sees: Crossmodal visual facilitation of human olfactory perception. Neuron. 2003; 39:375-386
Luisa Demattè M, Sanabria D, Spence C. Cross-modal associations between odors and colors. Chemical Senses. 2006; 31:531-538
McGurk H, MacDonald J. Hearing lips and seeing voices. Nature. 1976; 264:746-748
Shams L, Kamitani Y, Shimojo S. Visual illusion induced by sound. Brain Research. Cognitive Brain Research. 2002; 14:147-152
Haber SN. The place of dopamine in the cortico-basal ganglia circuit. Neuroscience. 2014; 282:248-257. DOI: 10.1016/j.neuroscience.2014.10.008
Vaughan RA, Foster JD. Mechanisms of dopamine transporter regulation in normal and disease states. Trends in Pharmacological Sciences. 2013; 34:489-496. DOI: 10.1016/j.tips.2013.07.005
Kägi G, Bhatia KP, Tolosa E. The role of DAT-SPECT in movement disorders. Journal of Neurology, Neurosurgery, and Psychiatry. 2010; 81:5-12. DOI: 10.1136/jnnp.2008.157370
Tatsch K, Poepperl G. Nigrostriatal dopamine terminal imaging with dopamine transporter SPECT: An update. Journal of Nuclear Medicine. 2013; 54:1331-1338. DOI: 10.2967/jnumed.112.105379
Jaakkola E, Joutsa J, Kaasinen V. Predictors of normal and abnormal outcome in clinical brain dopamine transporter imaging. Journal of Neural Transmission. 2016; 123:205-209. DOI: 10.1007/s00702-015-1495-0
Cerasa A, Gioia MC, Salsone M, Donzuso G, Chiriaco C, Realmuto S, et al. Neurofunctional correlates of attention rehabilitation in Parkinson’s disease: An explorative study. Neurological Sciences. 2014; 35:1173-1180. DOI: 10.1007/s10072-014-1666-z
Trujillo JP, Gerrits NJ, Vriend C, Berendse HW, van den Heuvel OA, van der Werf YD. Impaired planning in Parkinson’s disease is reflected by reduced brain activation and connectivity. Human Brain Mapping. 2015; 36:3703-3715. DOI: 10.1002/hbm.22873
Ventre-Dominey J, Bourret S, Mollion H, Broussolle E, Dominey PF. Dissociable dorsal and ventral frontostriatal working memory circuits: Evidence from subthalamic stimulation in Parkinson’s disease. Human Brain Mapping. 2014; 35:552-566. DOI: 10.1002/hbm.22205
Moustafa AA, Bell P, Eissa AM, Hewedi DH. The effects of clinical motor variables and medication dosage on working memory in Parkinson’s disease. Brain and Cognition. 2013; 82:137-145. DOI: 10.1016/j.bandc.2013.04.001
Alonso-Recio L, Serrano JM, Martín P. Selective attention and facial expression recognition in patients with Parkinson’s disease. Archives of Clinical Neuropsychology. 2014; 29:374-384. DOI: 10.1093/arclin/acu018
Sprengelmeyer R, Young AW, Mahn K, Schroeder U, Woitalla D, Büttner T, et al. Facial expression recognition in people with medicated and unmedicated Parkinson’s disease. Neuropsychologia. 2003; 41:1047-1057
Honma M, Kuroda T, Futamura A, Shiromaru A, Kawamura M. Dysfunctional counting of mental time in Parkinson’s disease. Scientific Reports. 2016; 6:25421. DOI: 10.1038/srep25421
Honma M, Murai Y, Shima S, Yotsumoto Y, Kuroda T, Futamura A, et al. Spatial distortion related to time compression during spatiotemporal production in Parkinson’s disease. Neuropsychologia. 2017; 102:61-69. DOI: 10.1016/j.neuropsychologia.2017.06.004
Doty RL, Deems DA, Stellar S. Olfactory dysfunction in parkinsonism: A general deficit unrelated to neurologic signs, disease stage, or disease duration. Neurology. 1988; 38:1237-1244
Masaoka Y, Yoshimura N, Inoue M, Kawamura M, Homma I. Impairment of odor recognition in Parkinson’s disease caused by weak activations of the orbitofrontal cortex. Neuroscience Letters. 2007; 412:45-50
Mahlknecht P, Pechlaner R, Boesveldt S, Volc D, Pinter B, Reiter E, et al. Optimizing odor identification testing as quick and accurate diagnostic tool for Parkinson’s disease. Movement Disorders. 2016; 31:1408-1413. DOI: 10.1002/mds.26637
Patin A, Pause BM. Human amygdala activations during nasal chemoreception. Neuropsychologia. 2015; 78:171-194. DOI: 10.1016/j.neuropsychologia.2015.10.009
Rolls ET. The rules of formation of the olfactory representations found in the orbitofrontal cortex olfactory areas in primates. Chemical Senses. 2001; 26:595-604
Braak H, Del Tredici K. Neuroanatomy and pathology of sporadic Parkinson’s disease. Advances in Anatomy, Embryology, and Cell Biology. 2009; 201:1-119
Chudler EH, Dong WK. The role of the basal ganglia in nociception and pain. Pain. 1995; 60:3-38
Schultz W. Reward functions of the basal ganglia. Journal of Neural Transmission. 2016; 123:679-693. DOI: 10.1007/s00702-016-1510-0
Middleton FA, Strick PL. Basal ganglia and cerebellar loops: Motor and cognitive circuits. Brain Research. Brain Research Reviews. 2000; 31:236-250
Reig R, Silberberg G. Multisensory integration in the mouse striatum. Neuron. 2014; 83:1200-1212. DOI: 10.1016/j.neuron.2014.07.033
Gentile G, Petkova VI, Ehrsson HH. Integration of visual and tactile signals from the hand in the human brain: An FMRI study. Journal of Neurophysiology. 2011; 105:910-922. DOI: 10.1152/jn.00840.2010
von Saldern S, Noppeney U. Sensory and striatal areas integrate auditory and visual signals into behavioral benefits during motion discrimination. Journal of Neuroscience. 2013; 33:8841-8849. DOI: 10.1523/JNEUROSCI.3020-12.2013
Honma M, Masaoka Y, Kuroda T, Futamura A, Shiromaru A, Izumizaki M, et al. Impairment of cross-modality of vision and olfaction in Parkinson disease. Neurology. 2018; 90:e977-e984. DOI: 10.1212/WNL.0000000000005110
Rolls ET, Baylis LL. Gustatory, olfactory, and visual convergence within the primate orbitofrontal cortex. Journal of Neuroscience. 1994; 14:5437-5452
Rolls ET. Top-down control of visual perception: Attention in natural vision. Perception. 2008; 37:333-354
Masaoka Y, Pantelis C, Phillips A, Kawamura M, Mimura M, Minegishi G, et al. Markers of brain illness may be hidden in your olfactory ability: A Japanese perspective. Neuroscience Letters. 2013; 549:182-185. DOI: 10.1016/j.neulet.2013.05.077
Doty RL, Stern MB, Pfeiffer C, Gollomp SM, Hurtig HI. Bilateral olfactory dysfunction in early stage treated and untreated idiopathic Parkinson’s disease. Journal of Neurology, Neurosurgery, and Psychiatry. 1992; 55:138-142
DeLong MR. Primate models of movement disorders of basal ganglia origin. Trends in Neurosciences. 1990; 13:281-285
Galvan A, Devergnas A, Wichmann T. Alterations in neuronal activity in basal ganglia-thalamocortical circuits in the parkinsonian state. Frontiers in Neuroanatomy. 2015; 9:5. DOI: 10.3389/fnana.2015.00005
Rey CD, Lipps J, Shansky RM. Dopamine D1 receptor activation rescues extinction impairments in low-estrogen female rats and induces cortical layer-specific activation changes in prefrontal-amygdala circuits. Neuropsychopharmacology. 2014; 39:1282-1289. DOI: 10.1038/npp.2013.338
Gattass R, Galkin TW, Desimone R, Ungerleider LG. Subcortical connections of area V4 in the macaque. The Journal of Comparative Neurology. 2014; 522:1941-1965. DOI: 10.1002/cne.23513
Chen MC, Yu H, Huang ZL, Lu J. Rapid eye movement sleep behavior disorder. Current Opinion in Neurobiology. 2013; 23:793-798. DOI: 10.1016/j.conb.2013.02.019
Cilia R, Marotta G, Benti R, Pezzoli G, Antonini A. Brain SPECT imaging in multiple system atrophy. Journal of Neural Transmission. 2005; 112:1635-1645
Rüb U, Seidel K, Heinsen H, Vonsattel JP, den Dunnen WF, Korf HW. Huntington’s disease (HD): The neuropathology of a multisystem neurodegenerative disorder of the human brain. Brain Pathology. 2016; 26:726-740. DOI: 10.1111/bpa.12426