Open access peer-reviewed chapter

Linking Neural Activity to Visual Perception: Separating Sensory and Attentional Contributions

By Jackson E.T. Smith, Nicolas Y. Masse, Chang'an A. Zhan and Erik P. Cook

Submitted: January 30th 2012Reviewed: April 16th 2012Published: September 26th 2012

DOI: 10.5772/48806

Downloaded: 1458

1. Introduction

For each of the five basic senses, information about the external world begins as a physical representation in the brain. This representation exists in the structure of sensory neural activity, such as the flow of ions across neural membranes and the action potentials (or spikes) that neurons produce. At some point the brain achieves a transition – from tangible electrophysiology to something more. In other words, neural activity becomes a basic sensation that we are aware of and that we can name. For example, sensations like ‘slow’ or ‘fast’, ‘far’ or ‘near’, are some of the simplest features that we can assign to a visual stimulus and are some of the basic attributes that we can perceive.

But the transition from neural activity to perception is not simple and remains largely unknown. This process is not intractable, however, and enormous effort has been made by neuroscientists to solve it. In particular, much progress has been made to reveal how small fluctuations in cortical activity are correlated with perceptual behavior. We refer to this correlation as ‘behavioral sensitivity’. New observations suggest that both bottom-up sensory mechanisms (such as neural noise) and top-down processes (such as attention) have a role to play in establishing behavioral sensitivity. How do we separate these two contributions?

Figure 1 illustrates the problem of untangling the link between a visual cortical neuron’s activity and a subject’s perceptual behavior. In the simplest model (Figure 1A), a visual cortical neuron contributes in a bottom-up manner to downstream networks that underlie perceptual behavior. In this case, a neuron is behaviorally sensitive because its activity is directly linked to the perception of the visual stimulus. In the alternative extreme (Figure 1B), a visual cortical neuron has no direct influence on the perceptual decision, but is

Figure 1.

Two neural mechanisms for a cortical neuron’s behavioral sensitivity. A cortical neuron exhibits behavioral sensitivity if its activity is correlated with perceptual behavior. For both mechanisms in A&B, information about the stimulus is encoded by visual cortical neurons and used to drive perceptual behavior. The arrows do not represent this flow of stimulus information; what they show are the source and destination of trial-to-trial variability. A, In the bottom-up mechanism, noisy sensory activity causes the variation in perceptual behavior. In this case, there is a causal link between the variability in visual cortical neurons and the variability in perceptual behavior. B, In the top-down mechanism, the subject’s attentional state varies from trial-to-trial, causing variable perceptual behavior. However, feedback projections also allow the attentional state to affect the firing rates of visual cortical neurons. In this case, there is a non-causal link between variability in visual cortical activity and perceptual behavior. C, Different sources of variation that could contribute to bottom-up sensory and top-down attentional variability in cortical neurons. Note that the bottom-up and top-down hypotheses shown here are the two possible extremes. The brain may actually implement any number of hybrid models, incorporating components from both hypotheses.

modulated by top-down attentional signals that affect both its activity and the perceptual behavior. In both models a neuron can theoretically exhibit the same behavioral sensitivity, but in the top-down scenario it has no role in the perceptual process. As we will discuss, both bottom-up sensory and top-down attentional mechanisms can be at play, depending on the perceptual task. To distinguish among the many possible contributions to a neuron’s behavioral sensitivity (Figure 1C), it is first important to understand the properties of behavioral sensitivity and how it is measured.

2. Area MT and visual motion perception as a model system

We can use the specialization of visual cortical neurons to begin understanding how they support visual perception. This is accomplished by comparing the activity of a neuron to the responses of an observer performing a perceptual task [1]. Neurons from the Middle Temporal area of visual cortex (MT i.e. area V5; [2]) are well suited to this purpose. In addition, the methods applied to the study of MT are generally applicable to other areas of visual cortex. MT is a part of the dorsal processing stream and it receives most of its sensory input from areas V1, V2 and V3 – while it sends output chiefly to parts of the parietal cortex [3]. In each hemisphere, area MT contains a complete topographical representation of the contralateral visual hemifield, and any one neuron receives visual information from a discrete patch of visual space, known as the neuron’s receptive field (RF). MT neurons are highly selective for both the direction and speed of visual motion, and produce crisp responses to preferred stimuli that fall within their RFs [4]; they are also selective for stimulus size [5] and binocular disparity [6].

Although V1, V2, and V3 neurons can also be selective for the direction and speed of visual motion, a relatively high proportion of MT neurons have an emergent Gestalt sensitivity to the motion of objects formed from separate components. For example, when shown two superimposed sine-wave gratings that move in different directions, V1 neurons mainly respond to the motion of only one component grating or the other; however, a number of MT neurons will treat the two gratings as a single object, responding to the coherent motion of both [7, 8]. Similarly, MT neurons can detect the coherent motion of separate dots – as if the dots were painted on an invisible pane of glass that was moving – even when the coherent dots are embedded within another field of dots that move randomly and incoherently [9].

Importantly, MT has been shown to take part in the perception of coherent dot motion – as lesioning MT causes a severe deficit in a subject’s ability to discriminate between opposite directions of motion [10], and microstimulating MT biases a subject to report motion in the preferred direction of the stimulation site [11]. Altogether, the robust responses and selectivity of MT neurons to visual motion, plus their involvement in motion perception, make them an excellent choice for comparison against the perceptual capabilities of a subject [1].

3. A neuron’s stimulus sensitivity

The classic studies of Newsome and colleagues demonstrated the power of a careful comparison between neural activity and perceptual behavior [9, 12]. Experiments were performed to carefully measure the discrimination sensitivity of MT neurons from monkey subjects performing a two-alternative, forced-choice (2AFC), motion-discrimination task. The subjects had to report whether the coherent motion in a patch of randomly moving dots was in the preferred or null (preferred + 180°) direction of an isolated MT neuron. It was critical to match the direction, speed, and location of dot motion to the neuron’s RF preferences. This ensured that the subject was responding to the same stimulus as the neuron. But more importantly: it maximized the chance that spikes recorded from the neuron were used by the subject to perform the task.

The direction of motion was randomly drawn on every trial so that the subject would have to watch the coherent dots carefully, in order to make a correct choice. However, the strength of coherent motion was also varied from trial to trial by changing the percentage of dots that moved together. This varied the difficulty of the task and therefore the subject’s performance, which provided a frame of reference. The neuron’s ability to discriminate the direction of coherent dot motion at any one difficulty level could be directly compared against the performance of the subject.

Figure 2.

Area under the receiver operating characteristic (ROC) curve. A, Hypothetical example of two spike-count distributions, from trials grouped by conditions X and Y. Spike counts range between 0 and the maximum value, cmax. B, The curved line, located above the dashed ‘chance’ line, represents the ROC curve that is constructed from the distributions in Panel A by classifying their values with the ideal observer (see Appendix). Classification performance is tested for every possible value of the classification criterion, c, which includes all possible spike counts between 0 and cmax. Thus, each value of c corresponds to a point in the ROC curve; the arrow shows how increasing values of c are mapped. The grey region is the area under the ROC curve. C, Behavioral sensitivity (or stimulus sensitivity) are defined as the area under the ROC curve that compares a distribution of failed-trial (or noise) spike counts (grey) versus a distribution of correct-trial (or signal) spike counts (open). The area under the ROC curve quantifies the difference between the two distributions.

A receiver operating characteristic (ROC) analysis (Figure 2) was used to quantify the discrimination sensitivity of MT neurons in the 2AFC task (see Appendix). For this, two distributions of spike counts were compared against each other, the distribution of counts from trials when the coherent motion was in the neuron’s preferred direction (distribution Y in Figure 2A) versus the distribution of counts from trials with coherent motion in the null direction (distribution X in Figure 2A). The resulting ROC areas (Figure 2B) described the probability that an ideal observer could tell which direction had been presented to the subject, based on the distribution of MT spike counts. This was computed separately for each level of coherent motion strength and compared directly against the subject’s performance. It was found that the average MT neuron could account for the subject’s discrimination sensitivity – at least under the particular conditions of the experiment [see 13].

The discrimination sensitivity of MT neurons in a 2AFC task is mirrored in experiments where the subject performs a slightly different task: motion detection. In such a task, the subject reports a change in the coherence of dot motion. The sensitivity of an MT neuron is judged by how different its firing rate is before and after the motion stimulus changes. Figure 3A shows an example motion detection task in which a monkey monitors a cloud of randomly moving dots (grey). At the start of each trial, all of the dots move independently (random dot motion) for a random duration. The subject was trained to release a lever in response to a brief (50 ms) period of coherent dot motion (motion pulse). Random dot motion resumed following the motion pulse. Trials were considered a failure if the subject did not release the lever following the coherent motion pulse. Importantly, the dots were located in the RF (dashed circle) of the MT neuron under study, and the coherent motion was always in the neuron’s preferred direction and speed. Again, this maximized the chance that the recorded spikes contributed to the subject’s performance.

The response of an example neuron – recorded from a monkey performing the motion detection task – is shown in Figure 3B. Each spike is represented as a black tick mark, and each row of ticks is the neuron’s response on one trial. Trials are sorted vertically by whether the subject was successful (correct trials, white background) or not (failed trials, grey background). In addition, correct trials are sorted by the duration between the motion pulse and the subject’s response time. All tick marks are aligned to the start of the 50 ms motion pulse (dashed line). Before the motion pulse, the neuron produced a baseline number of spikes in response to the random dot motion. Following the start of the motion pulse, however, the neuron responded with a vigorous burst of spikes. The stark contrast of the neuron’s responses show that it was very sensitive to the motion pulse.

The stimulus sensitivity of this example neuron is quantified using the ROC metric (Figure 2), similar to the one used by Newsome and colleagues. First, the spikes are counted on each trial in two analysis windows (black bars); one spans the 100 ms before the motion pulse (b), counting spikes produced in response to the random dot motion; the other spans the 100 ms after the burst of spikes began, counting spikes fired in response to the motion pulse (a). The distribution of spike counts from both windows are shown in Figure 3C. The neuron’s sensitivity to the motion pulse is found by comparing the distribution of spikes after the motion pulse (Figure 3C, open bars) versus the distribution of spikes before the motion pulse (Figure 3C, grey bars) using an ROC curve (refer to Figure 2). For this measure of stimulus selectivity, a value of 0.5 would indicate no difference between spike counts before and after the motion pulse, showing that a neuron’s response had no information about the visual stimulus. In comparison, values of stimulus selectivity approaching 0 indicate more spikes before the motion pulse, while values approaching 1 indicate more spikes after the motion pulse. As expected, the stimulus sensitivity of this neuron is very high (0.92), meaning that this neuron conveyed reliable information about the occurrence of the motion pulse.

Figure 3.

Stimulus and behavioral sensitivity of an example MT neuron. A, The perceptual task. A monkey directed its gaze to a fixation point (cross) and monitored a patch (grey) of randomly moving dots overlapping the neuron’s receptive field (dashed circle). At a random time, the dots moved coherently for 50 ms (motion pulse) in the neuron’s preferred direction and speed before reverting back to random dot motion. The trial was a success (correct trial) if the subject released a lever after the motion pulse. The trial was a failure if the subject did not respond. B, Raster of spike responses recorded electrophysiologically from an example MT neuron, while the animal subject performed the task in A. The analysis windows (a and b) were used to obtain the spike counts in C&D. C, The distribution of spike counts from before (window ‘b’ in panel B, grey) and after (window ‘a’ in panel B, open) the motion pulse used to obtain the ROC score that quantified the neuron’s stimulus sensitivity. D, The distribution of spike counts from failed trials (grey) and correct trials (open) after the motion pulse (window ‘a’ in panel B) used to obtain the ROC score that quantified the neuron’s behavioral sensitivity. Note that this neuron is exemplary and that few visual cortical neurons exhibit this level of behavioral sensitivity.

Whether the subject is detecting or discriminating motion, ROC analysis can be used to quantify the sensitivity of neurons to the stimulus; thus, both are referred to here as ‘stimulus sensitivity’. While lesion, microstimulation, and stimulus sensitivity studies show that MT is involved in motion perception and can account for its capabilities – they do not explain how MT activity becomes the perception of motion. This requires the estimation of the neural link to perceptual behavior, referred to as ‘behavioral sensitivity’.

4. A neuron’s behavioral sensitivity

The classic studies of Newsome and colleagues highlighted the large variation in the choices made by subjects and in the number of spikes fired by MT neurons. In response to statistically identical stimuli, subjects would sometimes report the wrong direction and their neurons would sometimes fire as if the opposite direction had been shown. However, this variation presented an exciting opportunity – because the ROC curve is a versatile tool and can be used to compare any two distributions of neural activity. Celebrini and Newsome [14] performed a ground-breaking analysis: they measured the correlation between the number of spikes fired by a neuron and the choice that the subject was about to make.

They began by grouping trials based on the ‘preferred’ or ‘null’ motion discrimination report made by the subject. Then they computed the ROC curve comparing the distribution of null-trial spike counts (distribution X in Figure 2A) versus the distribution of preferred-trial spike counts (distribution Y in Figure 2A). The area under this ROC curve is the probability that the ideal observer could correctly predict which direction of motion the subject would choose, using spike counts. This kind of ROC metric was named ‘choice probability’ when it was later used to analyze MT neurons [15]. However, we will refer to this ROC metric, and other like it, as ‘behavioral sensitivity’, because it measures how much the neural response predicts perceptual behavior. It is important to keep in mind that behavioral sensitivity does not measure the correlation between spike counts and perception itself – only the perceptual report, which may not always be faithful to what was actually perceived.

Similar to stimulus sensitivity, a behavioral sensitivity of 0.5 shows that there was no difference in the number of spikes fired prior to either choice (Figure 2C, left). If more spikes were fired prior to choices coinciding with the neuron’s preferred direction, then behavioral sensitivity would rise towards 1, to indicate a positive correlation (Figure 2C, middle and right). On the other hand, if more spikes were fired prior to null direction choices, then behavioral sensitivity would sink towards 0, to indicate a negative correlation. On average, MT neurons had a weak but significant, positive correlation with the subject’s upcoming choice of motion direction [15]. Since then, behavioral sensitivities have been observed between MT spike counts and the subject’s upcoming choice when discriminating coherent dot motion direction [16-18], speed [19, 20], disparity [21, 22], and cylindrical rotation [23-25]. Similar behavioral sensitivities have been observed between a subject’s discrimination performance and spike counts from cortical areas V2 [26, 27] and MST [14, 28, 29].

When subjects are tested on their ability to detect a change in coherent dot motion (Figure 3A), the ROC curve is made by comparing the distribution of spike counts from failed trials (distribution X in Figure 2A) versus the distribution of spike counts from successful trials (distribution Y in Figure 2A). The area under this curve is the probability that the ideal observer can correctly predict the subject’s detection performance, and so it was called ‘detect probability’ [13]. Again, we shall refer to this metric as behavioral sensitivity. A behavioral sensitivity above 0.5 indicates that the neuron fired more prior to successful detections – while behavioral sensitivity below 0.5 indicates that the neuron fired more prior to failures. Using behavioral sensitivity, correlations have been observed between MT spike counts and the subject’s ability to detect a change in coherent motion strength [13, 30] and speed [31], while similar behavioral sensitivities have been observed between a subject’s detection performance and spike counts from cortical areas V1 [32], V4 [33, 34], and VIP [13].

An example of how to compute behavioral sensitivity is shown for the same example MT neuron and task as before (Figure 3). This time, spike counts are only taken from the analysis window after the motion pulse (Figure 3B, bar ‘a’), but they are grouped by whether the trial was correct or failed. The distributions of spike counts on correct (open, Y in Figure 2A) and failed (grey, X in Figure 2A) trials is shown in Figure 3D. As this neuron was likely to fire more spikes on correct trials, its behavioral sensitivity was very high (0.88); thus, one could reliably predict the animal’s behavior from the neural responses.

5. Properties of behavioral sensitivity

The example neuron’s behavioral sensitivity (shown in Figure 3) is unusually strong. In general, the average sensitivity of visual neurons to perceptual behavior is much weaker. Table 1 lists the population averages over a number of studies; for most, it was under 0.6. Nevertheless, all averages were significantly greater than chance (0.5).

A typical behavioral sensitivity distribution for an example population of MT neurons is shown in Figure 4A. These neurons were recorded from two experiments, while monkeys performed either a motion detection [35] or a speed detection task [31]. These two experiments were combined because they were both detection tasks that used short, transient stimuli (~50 ms), as illustrated in Figure 3A. The mean behavioral sensitivity was weak, but significantly greater than 0.5 (mean = 0.54, two-sided t-test, p < 0.01). Therefore, behavioral sensitivity in visual neurons is a robust result, even though most neurons are only weakly sensitive to the subject’s upcoming behavior.

A second key observation is that neurons with high stimulus sensitivities are also highly sensitive to the subject’s perceptual behavior. A tempting interpretation is that the brain can determine which neurons are best able to support the subject’s performance on a task, then assign them a special role in guiding the subject’s behavior. To illustrate this relationship, we plot the distribution of stimulus sensitivities for the same example population of MT neurons in Figure 4B (mean stimulus sensitivity = 0.58, two-sided t-test, p < 0.01). To compare stimulus and behavior sensitivities, we first normalized each metric for each animal subject using a Z-score, in order to eliminate changes in the mean from affecting our

StudyTaskCortical AreaBehavioral Sensitivity
Liu and Newsome 2005 [19]Speed discriminationMT0.52
Gu et al. 2008 [28]Heading discriminationMST0.52
Sasaki and Uka 2009 [22]Direction discriminationMT0.53
Cohen and Maunsell 2010 [33]Orientation detectionV40.53
Cohen and Newsome 2009 [17]Direction discriminationMT0.54
Britten et al. 1996 [15]Direction discriminationMT0.55
Purushothaman & Bradley 2005 [16]Direction discriminationMT0.55
Law and Gold 2008 [18]Direction discriminationMT0.55
Price and Born 2010 [20]Speed discriminationMT & MST0.55
Nienborg and Cumming 2006 [26]Disparity discriminationV20.56
Sasaki and Uka 2009 [22]Disparity discriminationMT0.57
Bosking and Maunsell 2011 [30]Coherence detectionMT0.58
Smith, Zhan, and Cook 2011 [35]Coherence detectionMT0.58
Celebrini and Newsome 1994 [14]Direction discriminationMST0.59
Uka and DeAngelis 2004 [21]Disparity discriminationMT0.59
Herrington and Assad 2009 [31]Speed detectionMT0.59
Cook and Maunsell 2002 [13]Coherence detectionMT0.60
Palmer and Cheng 2007 [32]Gabor detectionV10.61
Herrington and Assad 2009 [31]Speed detectionLIP0.63
Dodd et al. 2001 [23]3D rotation discriminationMT0.67
Cook and Maunsell 2002 [13]Coherence detectionVIP0.70

Table 1.

able 1.Average behavioral sensitivity across different studies.

analysis. The results of plotting the normalized stimulus sensitivity versus the normalized behavioral sensitivity is shown in Figure 4C and illustrates a significant correlation (Spearman’s coefficient, R = 0.50, p < 0.01). The correlation between stimulus and behavioral sensitivity is an important property of visual neurons that is often observed [13-16, 18, 20, 21, 24, 28, 30, 35-38].

Figure 4.

Stimulus and behavioral sensitivity are correlated. A, The distribution of behavioral sensitivity for an example population of MT neurons, recorded during a motion detection task similar to that in Figure 3A. B, The distribution of stimulus sensitivity for the same population of neurons. In A&B, the height of each histogram bin shows the relative proportion of neurons with a sensitivity that falls within the bin’s range. C, The relationship between stimulus and behavioral sensitivity for the same population of neurons. Behavioral and stimulus sensitivity were normalized for each monkey so as not to introduce any spurious correlations when the data was combined. The Spearman’s correlation coefficient is R = 0.50 (p < 0.01). The best-fit, linear regression line is y = 0.46x + 0, where x is normalized stimulus sensitivity and y is normalized behavioral sensitivity. Data are combined from two experiments using four monkeys [31, 35].

The relationship of stimulus and behavioral sensitivity has, itself, two interesting properties. First, stimulus sensitive neurons seem to become behaviorally sensitive once the subject is well trained to perform the perceptual task [18, 37]; in fact, behavioral sensitivity only appears in neurons that can support the subject’s task strategy [21, 39]. Second, the correlation between stimulus and behavioral sensitivity tightens when attention is directed to the neuron’s RF (Nicolas Masse, unpublished observation). Altogether, these observations suggests that the most informative neurons are recruited by the brain to drive perceptual behavior.

There is evidence that the recruitment of informative neurons is a dynamic process, and adjusts to changing task demands. For instance, when two different types of motion-disparity stimuli are presented under different task conditions, the same MT neuron may show signs of involvement in the subject’s perception of one stimulus, but not another [25]. If the same task is used but the stimulus is presented through different modalities, only the MST neurons that respond in a similar manner to both modalities have strong behavioral sensitivities [28]. When the type of stimulus is consistent, behavioral sensitivity for the same MT neuron can vary as the direction of motion is dialed closer to or further from its preferred direction, in both a motion detection [30] or discrimination [16] task; a similar result holds for MST neurons during heading discrimination [29]. Lastly, when the type of stimulus is consistent but the subject performs two different perceptual tasks, behavioral sensitivity of the same MT neuron may be selective for the behavior on one task, but not the other [22]. These results suggest that the brain is constantly attempting to optimize the pool of visual neurons that it uses to drive perceptual behavior.

Several further properties of behavioral sensitivity are apparent in Table 1. Two of the studies examined neurons from different cortical areas using the same perceptual task ([13] coherence detection, MT & VIP; [31] speed detection, MT & LIP). Each study found that behavioral sensitivity was stronger in the areas further along the hierarchal processing stream (LIP and VIP). Similarly, a disparity discrimination study found behavioral sensitivity in V2 neurons but not in V1 [26]. Extensive investigation of somatosensory cortex has also shown that behavioral sensitivity grows along the hierarchal stream [40, 41]. These studies suggest that the closer a neuron is to downstream decision centers, the better its behavioral sensitivity becomes.

However, even lower level visual areas can demonstrate relatively strong behavioral sensitivity. Although V1 neurons had no sensitivity in a disparity-discrimination task [26], V1 is not thought to be directly involved in disparity perception [42]. On the other hand, V1 neurons are much better suited to supporting the perception of simpler stimuli, such as Gabor patches; accordingly, behavioral sensitivity emerges in V1 neurons when the subject detects Gabors [32]. Similarly, MT neurons are thought to participate in the perception of both motion [10, 11] and disparity [43]. When the subject discriminates a stimulus that requires the perception of both factors, MT neurons become more sensitive to the upcoming behavior [23]. Thus, the behavioral sensitivity of a neuron seems to reflect the extent to which it can support the subject’s perception of a given stimulus.

A critical observation is that the strength of behavioral sensitivity is strongly contextual [reviewed by 44]. When subjects are presented with ambiguous stimuli, MT neurons maintain sensitivity to the subject's upcoming behaviour, in 2AFC motion direction [15] and cylindrical rotation [23] discrimination tasks. Ambiguous stimuli carry no meaningful signal; that is, both directions of motion or rotation are equally well represented. In this case, the subject can make no correct choice based on the stimulus and is forced to guess. Although the ambiguous motion direction stimuli are not altogether different from the ambiguous rotation stimuli, MT neurons have much stronger behavioral sensitivities when the subject attempts to discriminate the latter. The main factor accounting for this is that the subject is looking for two-dimensional, linear motion in the first case – and three-dimensional, cylindrical rotation in the second.

A second effect of context is that behavioral sensitivity can strengthen over time on the same trial, following the onset of the stimulus [13, 21, 23, 27, 28, 30] but this is not always the case [15]. An important point to note is that behavioral sensitivity may rise even while the stimulus parameters remain constant. However, some results suggest that the duration of behavioral sensitivity is confined to the time in each trial when the neuron receives useful stimulus information [15, 21, 35].

One last contextual effect was demonstrated in a recent study of V2 neurons [27]; the subject’s motivation to perform a perceptual task was varied by changing the expected reward size. Smaller rewards, and therefore less motivation, were accompanied by a decrease in behavioral sensitivity.

A final property of behavioral sensitivity is that it persists when stimulus variation is removed. In both a motion direction discrimination [15] and motion detection [13] task, stimulus variation was removed by repeating the same random dot sequence on multiple trials. In conjunction with contextual effects, these results strongly suggest that behavioral sensitivity comes from mechanisms internal to the brain.

6. Bottom-up sensory versus top-down attentional contributions to behavioral sensitivity

The sensitivity of visual cortical neurons to the subject’s impending perceptual behavior is a robust result. Ironically, this fundamental observation has generated some controversy. The trouble is to explain which neural mechanisms produce behavioral sensitivity. Recently, two competing theories have emerged.

The first is the bottom-up hypothesis (Figure 1A). Formulated by Newsome and colleagues [15, 36], it was built upon a foundation of earlier results from psychophysics and neurophysiology, suggesting that the value of a perceived stimulus feature is coded by the collective firing rates from a population of sensory neurons [reviewed by 1]. A population is required when the stimulus responses of individual neurons are noisy; but, by averaging together the noisy responses of a population, an accurate representation of the stimulus can be obtained and used to drive behavior. Bottom-up sources of variation (Figure 1C) may include noise in the stimulus, noisy output from earlier stages of visual processing (e.g. retina, LGN), random eye movements [45], stochastic voltage channels [46], and autogenous noise from local networks [47, 48]. The central idea is that variability in perceptual performance comes directly from the noisy cortical representations of the stimulus.

In the context of the 2AFC direction discrimination task discussed above [12], Shadlen et al. [36] built a bottom-up model using two pools of noisy MT neurons; the first pool preferred motion in one direction, and the second pool preferred motion in the opposite direction. The responses of all neurons in a pool were summed together, and the two pooled signals were used by the model to asses the direction of motion: it chose the preferred direction of whichever pooled response was stronger. Because a direct, causal connection was established between the spike counts of each MT neuron and the model’s choice of direction, the model neurons had behavioral sensitivities above 0.5.

For purely bottom-up models, however, the impact of a sensory neuron on perceptual behavior should never change – nor should its behavioral sensitivity. And yet, behavioral sensitivity changes contextually depending on what the subject is looking for, how long the subject has viewed the stimulus, and the subject’s motivation level. A bottom-up model can not fully account for these observations, suggesting that high-order processes are involved in the behavioral sensitivity of neurons. The alternative top-down hypothesis (Figure 1B) was formulated to explain these results – in which signals are despatched to sensory cortex from high-order areas of the brain.

In this model, the subject’s attentional state varies from trial-to-trial, resulting in variable perceptual performance [see discussion of 15, 27, see review 44]; for example (Figure 1C), changes in spatial attention, prior expectation, motivation, or simply alertness can all affect the subject’s chance of success. If the same processes alter the firing rates of sensory neurons, then sensory spike counts would have a non-causal correlation with the subject’s performance. In other words, visual cortical neurons could exhibit behavioral sensitivity without actually affecting the perceptual behavior. Attention is a good example of a process that varies from trial to trial and affects both firing rates and perceptual performance [33, 34].

It is important to note that the bottom-up and top-down models, as described, are the two possible extremes at each end of a spectrum. For simplicity’s sake, they have been discussed separately. But the brain could implement a hybrid model [e.g. 37].

7. Evidence of bottom-up and top-down sources of behavioral sensitivity

Despite almost twenty years of study, there is no clear proof that one mechanism for behavioral sensitivity is entirely correct. The bottom-up hypothesis is attractive because it ties together a number of observations parsimoniously; neurons are weakly sensitive to the subject’s behavior when they are responding to an informative stimulus, when they are able to support perception of the stimulus features, and when they are able to support the subject’s task strategy. Furthermore, a neuron’s behavioral sensitivity scales with its sensitivity to the stimulus (Figure 4C). These properties are accounted for if the brain pools the output from a select set of informative neurons to form perceptual decisions, while ignoring output from uninformative neurons. The action of pooling offers a reason that the average behavioral sensitivity is weak: because the impact on behavior of any one neuron is diluted in the population response. However, the size of a neural pool required to form perceptions is unknown. There is some evidence that perceptual decisions are formed using only a few, highly informative neurons [38]. But the complication of correlated activity between neurons [49, 50] may require neural populations on a scale of hundreds [36].

A further attraction of the bottom-up hypothesis is that computational models with a bottom-up structure are able to emulate the subject’s behavior, neural responses and neural behavioral sensitivity, for a variety of tasks [13, 35-37]. Unfortunately, bottom-up models have difficulty explaining other properties of behavioral sensitivity without resorting to complex mechanisms. Although it would seem sensible that a subject should guide visual judgements using the most informative visual cortical neurons, it is uncertain how a purely bottom-up mechanism could recruit them. A simple way would be to change the synaptic weighting downstream of sensory areas, strengthening the connections from informative neurons and weakening the connections from uninformative neurons. This technique is able to alter the mean behavioral sensitivity of a simulated neural population [37]. Unexpectedly, synaptic weight changes alone cannot account for the correlation between a neuron’s stimulus and behavioral sensitivities (Figure 4C).

Correlations between the spiking activity of neurons can be the determining factor of behavioral sensitivity, especially for large pools of neurons [36]. Weak, inter-neural correlations have been observed throughout the brain [29, 49, 51-54] – with diverse implications [reviewed by 50]. Importantly, the level of correlation between neurons can inflate their behavioral sensitivities in two ways: in a bottom-up model, it reduces the independence of sensory neurons and tightens the covariance of any single one with the pooled response [36]; it can also cause neurons with no impact on perception to mimic other neurons that directly support perception [17, 55]. These effects of correlation on behavioral sensitivity increase for larger pools of neurons, while the effect of synaptic weighting decreases [36]. Thus, one cannot model the relationship between a neuron’s stimulus and behavioral sensitivity (Figure 4C) with synaptic weight changes, alone. However, if the correlation between two neurons is scaled by the similarity of their RF tuning and by the similarity of their stimulus sensitivity, then selectively weighting the more informative neurons can reproduce the observed relationship between stimulus and behavioral sensitivity [37]. Although an interplay between stimulus sensitivity, behavioral sensitivity, and the correlation between neurons is predicted, very little empirical verification has been published to date [24].

Bottom-up models also fail to explain the contextual variation of behavioral sensitivity. If top-down signals are able to selectively modulate the activity of targeted sensory neurons, then the top-down hypothesis is better placed to explain contextual variation, dynamic changes in behavioral sensitivity from trial to trial, and the relationship between stimulus and behavioral sensitivity. Attention [56] can modulate the activity of select neurons as well as affect the subject’s behavior; thus it is a good candidate for the source of top-down behavioral sensitivity. Trial-to-trial fluctuations in attention could equally as well have been the source of behavioral sensitivity in studies that otherwise supported a bottom-up model [15, 21, 35]. Furthermore, two recent multielectrode studies were able to estimate the level of attention on each trial, from the collective responses of many simultaneously recorded neurons [33, 34]; it was found that fluctuations in attention could account for the behavioral sensitivity of V4 neurons. Thus, a top-down mechanism is better able to account for some properties of behavioral sensitivity than a bottom-up mechanism.

8. Local field potentials and top-down mechanisms of behavioral sensitivity

Whether the behavioral sensitivity is due to top-down or bottom-up mechanisms depends entirely upon whether or not cortical neurons receive top-down signals in the first place. Specifically, attentional signals that predict the subject’s upcoming performance. But how can such signals be measured and separated from bottom-up visual inputs? When microelectrodes are used to record a voltage trace from the brain, they deliver far more information than just neural spiking activity. The short, discrete, high frequency waveforms of spikes ride on top of lower frequency changes in voltage.

Cortical local field potentials (LFP) are defined as the 1-200 Hz frequency bandwidth of the electrode voltage trace, and they result from the collective activity of neurons within approximately 250μm of the electrode tip [57, 58] – roughly the same spacial scale as a cortical column [59]. Because cortical tissue has no bandpassing effect on LFPs [60] and because their measurement is robust to the choice of microelectrode [61], LFPs are a clean way of taking neural activity measurements that are comparable across studies. Neurons generate LFPs in many ways [see review 62], including sub-threshold membrane activity and local oscillatory interactions between excitatory and inhibitory neurons; but an important source of LFP fluctuations are synaptic potentials in the dendrites: in other words, LFPs indirectly measure the input received by the local population of neurons.

Different sources of LFP fluctuations that operate at different time-scales can be partially isolated from each other by analyzing LFPs in the frequency domain [63]. The stimulus response of visual cortical LFPs from 3 to 90 Hz have been found to resemble the behavior of spiking responses from earlier processing stages [64-66], while higher frequency LFPs (>80 Hz) resemble the spiking responses of local neurons [67-71] – suggesting that LFPs below 80 Hz reflect synaptic activity. Upon closer inspection, it is found that LFPs from 1 to 12 Hz carry stimulus information that is independent of the stimulus information in LFPs above 40 Hz [72, 73]; but the band from 12 to 40 Hz does not carry stimulus information at all, raising the question of what its function is.

As visual neurons receive a substantial proportion of their input recurrently [74], LFPs must capture synaptic activity that results from neuromodulatory feedback. Generally, the spectral power of gamma band (40-80 Hz) LFPs is increased when attention is directed to the recording site’s RF [75-77], although it may decrease as well [78]. In area MT, this increase of gamma LFP power resembles the increase in firing rates of neurons in the same vicinity; however, beta band (12-24 Hz) LFP power from the same recordings behave differently in the face of attention – decreasing instead of increasing [79]. The decrease in low frequency LFP power is thought to result from the same attention related feedback that causes neurons to decorrelate from each other and reduce the variability of their responses [80]; this hypothesis is bolstered by the observation that the coherence of beta band LFPs from visual areas at different stages of hierarchal processing is strengthened by goal-directed attention [81, 82]. The relationship of beta band LFPs to top-down neuromodulatory signals – and their distinct response to them – make beta LFPs a good candidate measurement of the top-down input that arrives in sensory cortex.

The first demonstration that LFP spectral power is sensitive to the subject’s upcoming perceptual behavior was made from recordings in MT by Liu and Newsome [83]. Their innovation was to use distributions of spectral power rather than spike counts to compute the behavioral sensitivity of LFPs – by using a ROC curve to compare the distribution of power from trials when the subject reported the preferred stimulus of the recording site versus the distribution of power from trials when the subject reported the null stimulus. As a result, LFP behavioral sensitivity was a function of LFP frequency. Liu and Newsome observed that LFPs above 50 Hz had sensitivities above 0.5, indicating more spectral power when the subject was about to choose the preferred stimulus, which was similar to the positive spike-count behavioral sensitivities of MT neurons. Lower frequency LFPs, within the realm of alpha (8-12 Hz) and beta (12-24 Hz), showed a distinct behavior: they had sensitivities less than 0.5, which indicated less spectral power when the subject was about to choose the preferred stimulus. Since top-down neuromodulatory signals can explain this result, it may be the first demonstration that correlations between neural activity and perceptual behavior come from a top-down mechanism. However, it was also found that a trial-to-trial shift of LFP spectral power from lower frequencies to high frequencies could explain this result, without top-down signals.

9. Bottom-up and top-down processes work in sequence to produce behavioral sensitivity

Studies of behavioral sensitivity have had difficulty validating either the bottom-up or top-down hypothesis. But certain aspects of their experimental designs may have led to ambiguity. For example, many key studies have used long duration stimuli of 500 ms or more [13-16, 19-23, 25, 27, 30, 83]. This is problematic when trying to distinguish between a bottom-up and top-down mechanism – because it is impossible to tell when, or even whether, the neuron is responding to the stimulus or to top-down attentional signals. Recent studies, however, have begun to dissect bottom-up from top-down processes. They suggest that both mechanisms contribute to behavioral sensitivity, one after the other (illustrated in Figure 5).

Figure 5.

Bottom-up and top-down contributions to behavioral sensitivity are sequential. A, The order of perceptual events during the presentation of a long-duration stimulus, such as in [27]. The long duration stimulus allows a neuron’s behavioral sensitivity to be dominated by top-down contributions. B, The order of perceptual events following the presentation of a brief stimulus, as in [35]. Because the subject tends to respond before top-down contributions take effect, a neuron’s behavioral sensitivity is dominated by bottom-up contributions.

Nienborg and Cumming have demonstrated two contrasting processes by developing a psychophysical, reverse-correlation technique and comparing the results with the behavioral sensitivity of V2 neurons [27, 39]. Psychophysical reverse-correlation provided an estimate of the subject’s strategy when performing a disparity discrimination task. More importantly, it estimated when the subject was accumulating stimulus information. They found that subjects made the most use of stimulus information early in the trial – with later information being used progressively less, even though it was equally useful. In disagreement with a bottom-up prediction, they found that behavioral sensitivity moved in the opposite direction; it was weakest at the start of the trial, and then grew for approximately 500 ms before plateauing. A similar rise in behavioral sensitivity over time has been observed in other studies [13, 17, 20, 21, 23, 28, 30]. Together, these observations suggest an early, sensory accumulation process followed by a later process that was predictive of the subject’s upcoming behavior.

Rapid sensory accumulation and evaluation is necessary to explain any perceptual decision that is made before the late rise in behavioral sensitivity [84]. Accordingly, Nienborg and Cumming [27] observed behavioral sensitivity that was significantly greater than 0.5 within the first 500 ms of stimulation, when the subject made the most use of the stimulus information. But because their stimulus was long in duration and always appeared at the same time and location, it is impossible to know if the early component of behavioral sensitivity resulted from the early, sensory-accumulation process or from the late, behaviorally sensitive process (illustrated in Figure 5A).

Another recent study has looked exclusively at the behavioral sensitivity of MT neurons during early sensory accumulation, while subjects performed a motion-detection, reaction-time task [35]. This was achieved by using a very short (50 ms) motion signal that occurred at an unpredictable time and location. Unlike studies that used long stimulus presentation times, the neural sensitivity to behavior peaked approximately 100 ms after the motion signal began, during the short burst of spikes that neurons fired in response to the motion signal. Critically, both the transient neural response and the subject’s perceptual report (median RT = 400 ms) occurred well before the time in other studies that a late rise in behavioral sensitivity was established. Lastly, all aspects of the behavioral sensitivity time-courses and of the subject’s perceptual behavior were well accounted for by a purely bottom-up model. Together, these results suggest that sensory neurons have a bottom-up link to perceptual behavior during the early, sensory-accumulation process (illustrated in Figure 5B).

The late rise in behavioral sensitivity has parallels in other studies as well. In V1 neurons, spikes fired in response to a contour are able to distinguish whether it is the target or distractor stimulus, approximately 200–600 ms after the contour appeared [85, 86]. Similarly, the aperture problem is disentangled by MT firing rates approximately 150 ms after stimulus motion begins [8]. Of special interest is the recent evidence that stimulus information arrives in MT from outside the RF through top-down channels, approximately 400-500 ms following the onset of test stimuli in a match-to-sample task [87]. In these cases, late sensory neural activity carries information that was not originally available in the initial transient response, but likely arrives from a top-down source that has solved the problem at hand. These observations further support the idea that a top-down source of behavioral sensitivity engages sensory neurons following the initial transient response. While attention-like signals could fill the role of a top-down source following the start of the stimulus, it would have to allow for other attentional effects that are present earlier in the trial [33, 34, 52, 88].

10. Conclusion

To study the neural correlates of sensory perception in visual cortex, it is first necessary to understand how neural activity becomes selective for the upcoming perceptual behavior of a subject. This is referred to as a neuron’s behavioral sensitivity. There has been debate in the literature about whether the brain links sensory neural activity to perceptual behavior using a bottom-up or top-down mechanism. New results suggest that both mechanisms are active, but in a sequential fashion. The initial, transient responses of sensory neurons have a direct, bottom-up impact upon the subject’s behavior. Later responses reflect top-down signals that are linked to high-order processes, and attention fills the criteria necessary to drive top-down behavioral sensitivity. Although potentially difficult, new studies are needed to carefully distinguish and compare the contribution of both processes to the behavioral sensitivity of visual cortical neurons.

Acknowledgement

The authors wish to thank John Assad and Todd Herrington for the speed-pulse data used in Figure 4. The authors also acknowledge funding support from the Canadian Institutes of Health Research, The Natural Sciences and Engineering Research Council of Canada, and The EJLB Foundation.

Appendix: Area under the receiver operating characteristic (ROC) curve

The method used to quantify the discrimination sensitivity of MT neurons in a 2AFC task [9, 12] has since become a common technique of behavioral neurophysiology, and forms the basis of all major results reported in this study. The key question is whether the neurons fired more spikes in one condition than in another. Traditional parametric methods for answering this question place restrictive assumptions on the statistics of neural activity; for example, neurons do not always resemble a Poisson process [89]. Receiver operating characteristic (ROC) curves provide an unbiased, non-parametric way of quantifying the difference between any two distributions [90] – in this case, the number of spikes fired by a neuron on one set of trials versus another. Figure 2 illustrates how a ROC curve is used to quantify the difference between two distributions of spike counts (Figure 2A).

Faced with the problem of classifying a randomly sampled spike count as being from either distribution X (filled) or Y (open), the strategy adopted by the ideal observer is to choose a criterion level of c and assign any spike count less than c to X, and any spike count above c to Y. All possible values of c between 0 and the maximum spike count (cmax) must be tested to find the optimal criterion that makes correct classifications the most often.

To do so, a ROC curve is built by plotting the probability that spike counts sampled from X are greater than c against the probability that spike counts sampled from Y are greater than c (Figure 2B). When c = 0, all spike counts are greater than c, thus the beginning point of the ROC curve is always (1,1). As c is increased (Figure 2B, arrow) the performance of the ideal observer using that criterion level is plotted. When c hits cmax, no spike count is greater than c and the end point of the curve is (0,0).

The area under the ROC curve (Figure 2B, grey shading) is the probability that the ideal observer will correctly classify any given spike count from either distribution, and ranges between 0 and 1 accordingly; this probability is 0.75 for distributions X and Y from Figure 2A. Therefore, when X and Y are completely distinct from each other, the ideal observer correctly classifies 100% of all spike counts (area = 1, Figure 2C, right). On the other hand, if there is no distinction between X and Y, then the ideal observer has a 50% chance of correct classification – a coin toss (area = 0.5, left). If X and Y from Figure 2A switch positions then the difference between them remains the same; this is reflected by the area under the ROC curve, which is an equal distance below 0.5 after the switch (0.25 = 0.5 - 0.25, solid curve) as it was before (0.75 = 0.5 + 0.25, dotted curve).

Abbreviations

LGN – lateral geniculate nucleus
MT i.e. V5 – middle temporal area of visual cortex
MST – medial superior temporal area of visual cortex
VIP – ventral intraparietal area of visual cortex
LIP – lateral intraparietal area of visual cortex
2AFC – two-alternative forced choice
LFP – local field potential
RF – receptive field
ROC – receiver operating characteristic
RT – reaction time

Table 2.

Abbreviations

© 2012 The Author(s). Licensee IntechOpen. This chapter is distributed under the terms of the Creative Commons Attribution 3.0 License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

How to cite and reference

Link to this chapter Copy to clipboard

Cite this chapter Copy to clipboard

Jackson E.T. Smith, Nicolas Y. Masse, Chang'an A. Zhan and Erik P. Cook (September 26th 2012). Linking Neural Activity to Visual Perception: Separating Sensory and Attentional Contributions, Visual Cortex - Current Status and Perspectives, Stephane Molotchnikoff and Jean Rouat, IntechOpen, DOI: 10.5772/48806. Available from:

chapter statistics

1458total chapter downloads

More statistics for editors and authors

Login to your personal dashboard for more detailed statistics on your publications.

Access personal reporting

Related Content

This Book

Next chapter

Bio-Inspired Architecture for Clustering into Natural and Non-Natural Facial Expressions

By Claudio Castellanos Sánchez, Manuel Hernández Hernández and Pedro Luis Sánchez Orellana

Related Book

First chapter

Amygdala and Emotional Learning in Vertebrates – A Comparative Perspective

By Juan Pedro Vargas, Juan Carlos López and Manuel Portavella

We are IntechOpen, the world's leading publisher of Open Access books. Built by scientists, for scientists. Our readership spans scientists, professors, researchers, librarians, and students, as well as business professionals. We share our knowledge and peer-reveiwed research papers with libraries, scientific and engineering societies, and also work with corporate R&D departments and government entities.

More About Us