Audio Cortical Processing in Blind Individuals

Abstract

This chapter focuses on the cortical processing of auditory spatial information in blindness. Research has demonstrated enhanced auditory processing in blind individuals, suggesting they compensate for lacking vision with greater sensitivity in other senses. A few years ago, we demonstrated severely impaired auditory precision in congenitally blind individuals when performing an auditory spatial metric task: participants’ thresholds for spatially bisecting three consecutive, spatially distributed sound sources were seriously compromised. Here we describe psychophysical and neural correlates of this deficit, and we show that the deficit disappears if blind individuals are presented with coherent spatio-temporal cues (short space associated with short time and vice versa). Instead, when the audio information presents incoherent spatio-temporal cues (short space associated with long time and vice versa), sighted individuals are unaffected by the perturbation while blind individuals are strongly attracted to the temporal cue. These results suggest that blind participants use temporal cues to make audio spatial estimations and that the visual cortex seems to have a functional role in these perceptual tasks. In the present chapter, we illustrate our hypothesis, suggesting that the lack of vision may drive construction of multisensory cortical network coding space based on temporal instead of spatial coordinates.

1. Introduction: the development of space representation

The development of a multisensory space representation of the environment is crucial for humans to interact with objects and each other. Different sensory modalities represent space using varying reference systems: vision relies on retinotopic coordinates, audition on head-centered coordinates, and touch on body-centered ones. To perceive a multisensory world, human brains must combine the spatial information arriving from all the sensory modalities into a coherent representation. The visual modality seems to have a crucial role in this important step, and specifically in the process of developing an integrated multisensory representation of the environment. If vision is so important, then an obvious question arises: what happens to space representation when the visual input is missing? Studies of animals suggest that the lack of vision in the first period of life alters the development of space representation. For example, auditory spatial maps of juvenile barn owls change after visual adaptation with prismatic spectacles [1]. Likewise, total visual deprivation in young ferrets is associated with the development of disordered auditory spatial maps [2]. Similar transitory effects occur in humans. In a number of studies, auditory space representation altered after short periods of adaptation to non-aligned auditory and visual stimuli [3, 4]. In agreement with this idea, research shows the representation of the auditory space is dominated by visual experience among young children [5]. Taken together, these results support not only the idea that vision is important for developing auditory space representation, but also that its absence may interfere with such development.

2. Space representation and blindness

Since blindness represents a unique condition to investigate the role of the visual modality in the development of space representation, many researchers have investigated this topic. However, contradictory results have been found. Lack of visual experience is associated with an enhancement of auditory (e.g., [6, 7, 8, 9, 10, 11]) and tactile modalities [12] in blind compared with sighted individuals according to some studies. Results show that early blind subjects have enhanced skills in auditory pitch discrimination [13], localization of peripheral sounds in the horizontal plane [7, 10, 11], and ability to form spatial topographical maps underlying simple auditory localization [14, 15]. In particular, Lessard et al. [10] investigated the three-dimensional spatial mapping in early blind individuals by considering monaural and binaural listening conditions. Authors observed that early blind subjects show equal or better accuracy compared to sighted subjects when localizing single sounds. Moreover, they observed that early blind people could correctly localize sounds monaurally compared to sighted participants. Neurophysiological results indicate a clear response of the occipital cortex of blind individuals to auditory stimuli (e.g., [8, 16, 17, 18, 19]), revealing even topographic organization [20, 21, 22, 23, 24]. The absence of visual input also drives anatomical changes in the auditory cortex (e.g., [25, 26]). On the other side, other studies in humans and animals show that lack of vision is associated with spatial deficits. For example, studies show that blindness affects one’s ability to estimate the absolute distance of auditory cues [27, 28, 29], audio metric tasks [30, 31], auditory distance discrimination, and proprioceptive reproduction [32]. Research has also demonstrated poorer skills of blind compared to sighted people for localization of sounds along the mid-sagittal plane [11].

These results suggest that the mechanisms that subtend the development of space representation remain unclear. They also support that the role of the visual modality in space representation varies based on spatial properties, producing in some cases enhanced or impaired skills in blind individuals. The mechanisms behind this require clarification.

3. Audio metric impairment in blind individuals

Another exception of the enhanced skills of blind individuals in space representation is the ability to perform an audio spatial bisection task [30]. Contrary to previous works studying pitch and timbre discrimination [13, 33], or localization of single sounds in space [7, 10], the bisection task requires estimation and comparison of different locations in space. While sighted children of 6 years of age can perform it [5], our study found that blind individuals were strongly impaired in this task. The results were in agreement with previous findings of our group showing that, during development, the visual modality dominates the multisensory spatial percept in audio-visual conditions of the bisection task, suggesting that the visual input might be crucial for the development of audio spatial bisection skills [5]. During the task, participants sat 180 cm from the center of a bank of 23 speakers and perceived three sounds: the first speaker (on the left) and the last speaker (on the right) delivered the first and the third sounds, respectively. The second sound came from an intermediate speaker between the first and the last one (see Figure 1). Participants verbally reported whether the second sound was closer to the first (i.e., left) or to the last (i.e., right) sound.

Figure 1.

Description of the spatial bisection task. Participants were aligned with the central speaker (i.e., 0°) and listened to a sequence of three sounds. The first and the third sound were delivered from the first speaker on the left (i.e., −25°) and the last speaker on the right (i.e., +25°) respectively, whereas the second sound derived from an intermediate speaker between the first and the last one (i.e., between −25° and +25°). Participants were asked whether the second sound was closer to the first (i.e., left, −25°) or the last (i.e., right, +25°) sound. Upper panel reports an exemplar trial in which the second sound is closer to the last (i.e., right) sound.

While sighted individuals succeeded at the task, with responses varying systematically as a function of speaker position (a standard deviation of 4.3°), blind individuals provided almost random responses. However, the same deficit was absent for other tasks, such as the minimal audible angle task in which participants were asked to evaluate which sound was from the left in a sequence of two sounds (this result is in agreement with previous studies [10]). The deficit we reported for the bisection task was far larger than the perceptual enhancements that have been reported before, and it was highly consistent among blind individuals. Figure 2 reports the individual thresholds for the minimum audible angle against individual thresholds for the bisection task: the thresholds of blind individuals are over the equality line. While the study observed no difference between groups for the minimum audible angle (t test, p = 0.21), groups significantly differed for the bisection thresholds (Wilcoxon signed ranks test, p < 0.01; bootstrap sign-test: p < 10^–5). We performed other tests we do not report here so as to show the specificity of the deficit and its independency from the kind of sound used (e.g., the pointing task; for more details, see [30]). We also performed a temporal version of the bisection task, in which the participants performed the same task in the temporal domain. Participants had to report if the second sound was closer to the first or to the last in time. In this task, no deficit emerged, suggesting that the deficit was not due to a general/aspecific impairment, to task incomprehension, or to attention and memory problems associated with task difficulty.

Figure 2.

Individual data, plotting bisection thresholds against minimal audible angle. Arrows at the margin show the geometric means of each group as well as the shaded areas of 95% confidence intervals. The blue and green arrows show the average thresholds for 7- and 10-year-old children, respectively (taken from a previous study) [5]. The dashed diagonal line is the equality line: while the thresholds of sighted subjects are scattered around this line, all except one non-sighted subject are above it. Indeed, the only non-sighted subject with bisection threshold that falls within the control range had a threshold for minimal audible angle that was six times lower than the mean of the controls, meaning the subject’s data point sits well above the bisection line. With permission from Gori et al. [30].

4. Cortical processing of space and blindness

Scientific evidence suggests that the auditory and somatosensory systems colonize the visual cortex of congenitally blind individuals to a certain extent (e.g., [16, 34]). For example, studies that were performed with fMRI [35, 36, 37, 38] and event-related potentials (ERPs [39, 40]) show that the visual cortex shows a strong and reliable response to sound presented alone. Tomasello et al. [41] have recently proposed a neurocomputational model to explain the visual cortex recruitment during language processing in congenitally blind individuals. For what concerns space representation, Collignon and colleagues [42] compared the brain activity of early blind and sighted individuals during a spatial and pitch task using the same stimuli for both. Authors observed that the processing of sounds recruited the occipital cortex, and the spatial processing of audio spatial stimuli also activated the dorsal occipital stream involved in visuospatial/motion processing in sighted individuals. They concluded that some regions of the right dorsal occipital stream specialize toward processing spatial information without the necessity of visual experience. Not only are visual areas activated during auditory tasks, but also localization abilities of blind subjects are strongly associated with the magnitude of visual cortex activity [8, 43, 44]. For example, early blind people localize sounds more accurately than those who are sighted under monaural conditions [10]. Their activation in right-hemisphere striate and ventral extrastriate areas correlates with the performance in a pointing task to monaurally presented sounds [8]. These results suggest that the enhancement of some auditory skills of blind individuals may reflect in the recruitment of the visual cortex. From this arises the question: what about the impaired skills, such as in the case of the bisection task? If the visual information is important for the development of audio space bisection [45], as we reported in the previous section, then we may expect the visual cortex of sighted and not of blind individuals [30] should be recruited for this audio processing. We recently used EEG to measure activation of the occipital cortex of sighted and blind individuals during the audio bisection task [45, 46]. Figure 3 illustrates the scalp maps elicited by the second sound of the spatial bisection task when it was delivered from the left (i.e., −4.5°, see left panel) and the right side (i.e., +4.5°, see right panel) in the 50- to 90-ms time window after sound onset, for sighted (Figure 3A) and blind participants (Figure 3B). In the case of both groups, two strong positivities emerged: one involving central areas and one involving parieto-occipital areas. However, the latter positivity showed a specific contralateral pattern during the spatial bisection task that was only in sighted subjects (Figure 3A). In early blind participants (Figure 3B), the parieto-occipital response was strongly attenuated and not contralateral to the sound spatial position.

Figure 3.

Scalp maps of the mean ERP amplitude in the selected time window (50–90 ms) after the second sound of the spatial bisection task, for sighted (A) and blind (B) groups. Left and right panels of the figure report the conditions in which S2 was presented from either −4.5° (i.e., narrow first distance) or + 4.5° (i.e., wide first distance), respectively, and independently of timing (±250 ms). With permission from Campus et al. [46].

To provide evidence that the early contralateral component that we observed over the occipital scalp actually involved generators in occipital areas, we performed comparisons between groups at the source level (Figure 4). Results suggest that sighted subjects showed a stronger occipital and temporal activation contralateral to the physical sound position, while early blind subjects exhibited reduced activation in contralateral cortical areas and an increased activation in ipsilateral cortical areas.

Figure 4.

Average source activity within the selected time window (50–90 ms) compared between sighted and blind subjects. Left and right panels of the figure report the conditions in which S2 was presented from either the left (i.e., −4.5°, narrow first distance) or the right side (i.e., +4.5°, wide first distance), respectively. We report results of paired two tailed t tests with the scale in terms of t-statistic. We also display significant values of t statistic: reddish and bluish colors indicate stronger activations in sighted and early blind subjects, respectively, while intensity indicates magnitude of t (i.e., strength of difference). Only t values corresponding to p < 0.0001 after FDR correction appear. Adapted with permission from Campus et al. [46].

In early blind individuals, the laterality was absent, which means that early visual experience mediates development of this contralateral early occipital response. The data suggest that visual modality plays a key role in the development of an early occipital response that is specific for space perception and auditory stimuli. In sighted subjects, the acoustic recruitment of the visual brain may be necessary to build a spatial metric of the environment using high resolution and flexibility that only the visual brain is capable of implementing. Lack of vision seems to impact the development of this processing and underlying neural circuits, thereby impairing understanding of Euclidean relationships, such as those involved in solving a spatial bisection task. These findings agree with our previous behavioral results [30], at the same time revealing that the neural correlates of the audio space bisection deficit reported in blind individuals might correspond to reduction of early occipital contralateral activation. We speculate that cortical activation underlying the C1 ERP component (usually elicited by visual stimuli) plays a fundamental role in the construction of metrics in the spatial domain independently of the involved sensory modality. Moreover, the construction of spatial metrics may depend on visual experience.

5. Blindness duration and cortical reorganization

The lack of vision seems to interfere with the development of space representation, so an interesting question is: what happens when the subject loses visual input later in life? Late blindness is a condition worthy of investigation concerning this issue because spatial hearing of late blind subjects is shaped by unique combination of visual calibration in childhood and prolonged blindness in adulthood. As well as for early blind individuals, research on late blind individuals shows contrasting results at both the behavioral and cortical levels. For example, scientific evidence shows that, late blind individuals are better compared to sighted people in using spectral cues when they localize sound position in peripheral regions [47, 48]. Similar to early blind participants, they also show auditory and tactile recruitment of occipital regions [49, 50, 51]. Voss and colleagues [43] investigated the effect of blindness on brain activity by using positron emission tomography (PET) during one binaural and one monaural sound source discrimination task (SSDT) in early and late-onset blind individuals. In their study, Voss et al. observed that no difference was present between groups for the binaural task. Contrarily, during the monaural condition, early blind individuals performed significantly better than all the other groups (in agreement with the behavioral study of Lessard and colleagues [10]). Late blind subjects are more similar to sighted individuals concerning other skills too, such as absolute auditory distance estimation [27], locational judgments after a perspective change in small-scale space [52], audio shape recognition and navigation tasks [53]. In a recent study from our group [54], late blind individuals were involved to allow the study to investigate how blindness duration (BD) affects auditory spatial bisection skills and neural correlates. In late blind individuals, we replicated the same behavioral and EEG experiment previously performed among early blind people (see section above, [46]). We observed that the early (50–90 ms) ERP response, previously observed in sighted [45] and not in early blind [46] individuals, is dependent on the amount of time spent without vision (i.e., BD, [54]). In particular, we observed that a shorter period of blindness links to stronger contralateral activation in the visual cortex (see Figure 5) and better performance during spatial bisection tasks. Contrarily, we observed non-lateralized visual activation and lower performance in individuals who had experienced a longer period of blindness.

Figure 5.

Results of linear regression analyses in late blind individuals. Years of blindness duration (BD) negatively correlate with lateralized (i.e., contralateral-ipsilateral to S2 position) ERP amplitude in a 50- to 90-ms time window after S2 for the spatial bisection task. With permission from Amadeo et al. [54].

Time spent without vision seems to gradually impact neural circuits underlying the construction of space representation in late blind participants. On the one hand, duration of blindness directly impacts both neural and behavioral correlates of late blind individuals during auditory spatial processing similarity between neural circuits and competences of late blind individuals with short blindness duration and, confirming there is indeed a key relationship between visual deprivation and auditory spatial abilities in humans. On the other hand, the sighted people suggest that an early visual experience is necessary and sufficient to fully develop neural areas involved in complex representations of space. These results agree with previous works in animals showing visual information during the first years of life is essential toward calibrating auditory space representation in the brain [5, 30, 31, 53, 55].

6. Time to infer space in blindness

Almost 100 years ago, Piaget [56] stated that the temporal metric is strictly related to spatial metric development: “Space is a still of time, while time is space in motion” [57]. What Piaget did not discuss is the role of different sensory modalities in this link. Visual experience is important for the development of spatial metric representations, such as for bisecting sounds. Starting from Piaget’s idea, one might hypothesize that when vision is unavailable, such as in the case of blindness, temporal representation of events can set spatial representation. Indeed, while early and late blind individuals with long blindness duration show strong deficits in terms of spatial bisection tasks, they show performance and cortical activations similar to sighted individuals in the time domain, such as in a temporal bisection task [45]. In support of this hypothesis, we recently tested and verified that space representation of blind individuals is strongly influenced by the temporal representation of events [58]. We performed different versions of the spatial bisection task in sighted and blind individuals, in which we presented spatial and temporal independent, coherent, and conflicting information (Figure 6 top panel). Similar to the original version of the bisection task [30], in one condition the temporal delay between the three sounds was always the same, and only spatial cues were relevant to compute the task (i.e., Equal bisection, Figure 6A top panel). In other conditions instead, we presented a spatio-temporal coherent or conflicting information. For example, in the coherent bisection, a longer spatial distance between the first and the second sound was associated with a longer temporal delay between the two sounds, and the reverse was the case for shorter distances (see Figure 6B top panel). In the opposite bisection, a longer spatial distance between the first and the second sound was associated with a shorter temporal delay between the two sounds, and the reverse for shorter distances (see Figure 6C top panel). Thanks to these two manipulations, it was possible to disentangle the role of spatial and temporal cues when it comes to the audio spatial bisection task. Our results show that these manipulations modified the performance of blind but not sighted participants. Indeed, in blind individuals, the spatial bisection deficit observed in the original version of the task disappeared when the study presented coherent temporal and spatial cues, and it increased in the conflicting condition. Figure 6 (lower panels) plots the proportion of answer “second sound closer to the third sound” as a function of the position of the second sound for one blind (in red) and one age-matched sighted individual (in gray). In the equal bisection condition, we observed the same deficit observed previously [30], with random responses and no psychometric function for the blind subject. Interestingly, in the coherent bisection condition, the deficit disappeared and there was similar performance between the sighted and blind participants. More interestingly, in the opposite bisection condition (Figure 6C), while there was no effect of the manipulation that was evident in the sighted individual, in the blind individual, the response was inverted (i.e., the psychometric function was reversed and presented in the opposite direction than expected).

Figure 6.

Bisection tasks: coherent and conflicting manipulations of space and time. Results of the three conditions of the spatial bisection task for a typical blind participant (red symbols) and a typical sighted control (gray symbols). Subjects sat in front of an array of 23 speakers, which are illustrated by the sketches. (A) Equal spatial bisection. Top: the time interval between the first and the second sound (750 ms) was equal to the time interval between the second and the third sound. Bottom: proportion of trials judged “closer to the right sound source” plotted against the speaker position for the second sound. The size of the dots is proportional to trial number at that position. We fitted both sets of data with the Gaussian error function. (B) Coherent spatial bisection. Top: spatial distances and temporal intervals between the three sounds were directly proportional (e.g., long spatial distance and long temporal interval). Bottom: same as for (A). (C) Opposite spatial bisection. Top: spatial distances and temporal intervals between the three sounds were inversely proportional (e.g., long spatial distance and short temporal interval). Bottom: same as for (A) and (B). With permission from Gori et al. [58].

Performance of blind individuals reveals a strong temporal dominance for the spatial bisection task, suggesting that temporal cue is attracting the spatial auditory response [58]. A possible explanation is that, while the retinotopic organization of the visual cortex may support the reorganization underlying some enhanced audio spatial skills in blindness (such as the sound localization ability), it may be insufficient to guarantee the development of more complex spatial skills, such as those required for the audio spatial bisection task. Our results about the role of time in space representation suggest that temporal information can act as an alternative cue for reorganizing space representation subtending some more complex spatial abilities.

7. Space, time, and speed

How can temporal information support space processing in blindness? It might be that, for some complex spatial representations, the visual system calibrates the auditory sense of space by processing the speed of the stimuli. Neurons that process speed information have been demonstrated for the visual modality in the visual cortex [59]. These neurons could be responsible for processing information during spatial bisection tasks.

In typical conditions, it may be that the visual system facilitates transfer of audio processing from a temporal to a spatial coordinate system. Indeed, audition is the most reliable sense to represent time information, and vision is the most reliable sense to represent space information. The mediator between auditory time and visual space could be velocity processing, which may represent a channel of communication between the two sensory systems. Figure 7 reports a graphical description of how vision and audition may collaborate to estimate space and time starting from the speed properties of an object. Concerning space estimation in sighted individuals, given the higher weight of vision, it is independent of the temporal coordinates of the stimulus for both coherent (Figure 7A) and conflicting (Figure 7B) situations. On the other hand, when the visual information is unavailable, the spatial counterpart seems unable to develop and blind individuals seem to rely only on temporal coordinates to infer metric spatial information. One might then speculate that when the visual network is impaired, blind individuals internalize a statistical prior (i.e., a prior on the constant velocity of stimuli) derived from environmental statistics. This drives them to infer space from time. This idea is in agreement with the Imputed Velocity Theory [60], which asserts that humans intuitively attribute constant velocity to a single object moving through space over time. If we assume that blind individuals assume a prior of constant velocity of objects in space, they can use this information to extract space cues using time cues. This strategy would help blind people to overcome metric problems by using unimpaired temporal maps to decode spatial metrics. This may also facilitate their interaction with others (Figure 7 left). This mechanism would be adaptive for blind individuals as it allows them to process spatial information correctly at the auditory level based on its temporal representation. On the other side, this mechanism could be maladaptive when conflicting spatial and temporal information is provided, as blind individuals can be deceived by the temporal cue in the spatial evaluation, perceiving an illusory spatial position of the sound based on its temporal coordinates (Figure 7 right).

Figure 7.

Graphical model of our theory. In sighted individuals, spatial estimation is independent of the temporal cue of the stimulus for both coherent (A) and conflicting (B) information. Blind individuals infer spatial information using temporal coordinates of the stimulus assuming constant velocity. When spatial-temporal coherent stimuli are present, the spatial estimation can be successfully extracted by the temporal cue (C). On the other hand, when conflicting spatial-temporal information (D) is provided, the temporal cue is wrongly used to drive the spatial sound position assuming constant velocity.

These findings support the cross-sensory calibration theory [5, 61], suggesting that visual information is necessary for normal development of auditory sense of space. In children younger than 12 years of age, there is visual dominance over audition in spatial bisection, and an auditory dominance over vision in temporal bisection [5]. The cross-sensory calibration of the visual system for the spatial bisection explains why blind subjects show a specific temporal response to the spatial bisection task, while also showing different processing to solve Euclidean, metric, relationships. We can speculate that these processes could be mediated in sighted but not in blind people by pathways involving the superior colliculus [30, 55, 62]. The present study adds new evidence, showing other possible interactions during development among sensory modalities, as well as spatial and temporal domains.

8. Conclusion

A lack of vision hampers strategies and neural circuits underlying complex spatial metrics, driving to multisensory interactions that bring to code space based on temporal instead of spatial coordinates. These findings open new opportunities for developing sensory substitution devices and rehabilitation technologies for blind people, where spatial and temporal cues could be simultaneously manipulated to convey richer information.

Acknowledgments

The authors acknowledge the contribution of David Burr and Concetta Morrone in helping to shape the line of research that forms the basis of this chapter, including their continuous support and their participation in some of the experiments reported.

Conflict of interest

The authors declare no conflicts of interest.