Altered Associative Learning and Learned Helplessness in Major Depression

1. Introduction

Major depressive disorder (MDD) is a highly prevalent and recurrent disorder. Recent epidemiological studies have shown that up to 16% of the general population will suffer from at least one clinical episode of depression in their lifetime with women being affected more frequently than men (for a review, see [1]). The World Health Organization considers depressive disorders as one of the leading causes of disease worldwide, accounting for about 4.4% of total disability adjusted life years (DALY; [2]). Longitudinal studies indicate that up to 85% of depressed patients suffer from multiple episodes [3], and that 15-20% of episodes take a chronic course [4]. However, a unitary model providing axiomatic factors related to the development and maintenance of depression has not been established so far, what is most likely due to substantial heterogeneity in the etiology and symptomatology of depressive syndromes [5].

This chapter aims to provide a selective review of evidence on how alterations in associative learning relate to the (etio-) psychopathology of depression in the context of widely accepted models of the disorder.

2. Models of depression

The literature on the development and maintenance of MDD is characterized by several lines of research that have highlighted alterations on different levels (i.e., cognitive-emotional, behavioral, and psychophysiological) to be relevant for the understanding of depression. Cognitive models of depression focus on alterations in human information processing by investigating attributional style and other cognitive variables, recently also including rumination. Behavioral and neurobiological models of depression dominantly refer to animal models of depression such as chronic stress or learned helplessness to investigate behavioral, endocrinological, and molecular characteristics of depression-like behaviors. Meanwhile, recent neuroimaging studies in humans aim to isolate specific structural and functional alterations in the brain associated with dysfunctions of emotion, motivation, and cognition in depression.

The current diagnosis of depressive disorders according to recent versions of the Diagnostic and Statistical Manual of Mental Disorders (DSM-IV) [6] and the International Statistical Classification of Diseases and Health Related Problems (ICD-10) [7] exclusively refer to the presence of decisive symptoms including depressed mood and anhedonia in a given time period. Although these nosological classification systems allow for objective diagnosis of and communication about depression, the neuropsychopathological signature of the disorder as proposed by depression models is not leading for the diagnosis. As a consequence, current depression diagnosis and research on the psychological and psychophysiological correlates of depression do not inevitable fall into place.

2.1. Cognitive models of depression

Cognitive theories of depression are among the most prominent models of depression. The two most influential cognitive theories, Beck’s schema theory and the reformulated hopelessness theory by Abramson and colleagues (cf., [8]), point to the role of maladaptive self-schemata and negative inferential style for the onset, course, and outcome of depression. Prospective studies have shown that negative attributional styles and dysfunctional attitudes predict the onset of depressed mood and symptoms [9]. Parallel research strategies using experimental paradigms from cognitive psychology found depression to be associated with excessive attending to negative stimuli, fast recall of negative memories, over-generalized recall of autobiographical experiences, and a tendency for negative judgments on hypothetical and real-life experiences (e.g., [10]).

A third line of research deals with cognitive processes of affect regulation that might predict recovery from or worsening of depressed mood. In this context, the response styles theory by Nolen-Hoeksema [11] addresses the role of perseverative self-focused rumination versus distraction from negative mood for the exacerbation, maintenance, and discontinuation of depressed states. Ruminative responses are defined as thoughts and behaviors that comprise passively focusing one’s attention on one’s depressive symptoms, and repetitively thinking about possible causes and consequences of these symptoms. Distractive coping is defined as actively turning one’s attention away from one’s symptoms to pleasant or neutral thoughts and actions. There is strong evidence from laboratory and observational studies with nonclinical samples for the proposed predictions of response styles, particularly rumination, on the severity and duration of depressed mood. Self-reported rumination is associated with depression severity [12-14] and experimentally induced rumination prolongs dysphoric mood, enhances negatively biased memories, and impairs interpersonal and complex problem solving, while induced distraction predicts the decline of depressed mood [15]. Furthermore, a ruminative response style was found to predict future levels of depressed mood, even after controlling for baseline levels of depression (e.g., [11, 13-14]). High trait rumination scores in currently and past depressed subjects point to the role of a ruminative style as a potential vulnerability factor for depression [16-17].

These findings generally support the hypothesis that depressed individuals suffer from two dominant cognitive biases [18]: First, depressed individuals show increased attention for negative information; and second, they show extensive self-referential processing concerning the (negative) appraisal of stimuli and experiences.

2.2. Behavioral and neurobiological models of depression

Animal models of depression have dominantly focused on paradigms such as chronic mild stress or learned helplessness to induce depression-like behavior in unselected strains of animals or in animals bred for susceptibility to stress. When exposing animals to inescapable shocks or chronic mild stress they show subsequent impairments in active escape responses and a reduction in responsiveness to rewards as well as distinct neuroendocrinological changes [19-21]. These models in addition to lesion studies in animals [22] have generated many hypotheses about the neurobiological mechanisms involved in depression [23]. In parallel, proposed alterations in candidate regions and neural networks, assumed to play a major role in depression, have been found in neuroimaging studies in humans. Besides structural alterations mainly in terms of reduced grey-matter volumes in fronto-limbic regions [22], functional alterations in frontal regions, including the anterior cingulate cortex, and limbic structures, such as the amygdala and hippocampus, have been detected in prominent functional imaging studies. Recently, functional alterations in striatal structures (nucleus accumbens, caudate, putamen) have been related to altered reward and loss processing in depression [24]. Meta-analytic findings [25] emphasize that depression is characterized by predominantly reduced activity in the rostral anterior cingulate cortex and anterior insula, which is linked to altered salience processing of emotional and cognitive stimuli. In addition, hypersensitivity to negative information and reduced executive functioning seem substantially associated with a lack of prefrontal control in terms of both exaggerated frontal hypo- and hyperactivity. Functional alterations in striatal regions seem closely related to biased valence processing in MDD with a hemispheric dissociation depicting right-sided hypoactivity to positive and left-sided hyperactivity to negative stimuli. Moreover, increased activation in an extended medial prefrontal network during self-referential processing was found in depressed individuals [26].

Thus, biased information processing in depression as proposed by cognitive models of the disorder obviously correlate with partly specific neurofunctional alterations in depressed individuals. Several lines of evidence point to medial prefrontal, limbic, and striato-pallido-thalamic regions to be critically involved in the pathophysiology of MDD [27]. However, it needs to be mentioned that rather heterogeneous than homogeneous results for multiple cortical and subcortical regions characterize the current state on functional neuroimaging findings in major depression [25].

2.3. Integrative diathesis-stress models of depression

In attempting to understand how alterations on the emotional-cognitive and neurophysiological dimension emerge, diathesis-stress models generally postulate that both biological and environmental factors affect the development of psychological disorders, including depression [28]. The basic assumption is that stress activates a diathesis in turn transforming the predisposition or vulnerability for a disorder into the presence of the disorder. During the long history of this general model, the concepts for vulnerability and stress have notably changed. Importantly, although multiple events might be universally termed as stressful (e.g., the death of a significant person), the experience of stress is assumed to be dependent on the individual’s appraisal of negative events. Likewise, the concept of vulnerability – initially focusing on heritable and biological factors – has been enriched by including psychological factors, such as cognitive and interpersonal variables [28]. As a consequence, the rigorous distinction between external (stressors) and internal (vulnerability) factors has been abandoned in support of an interactive perspective. That is, the diathesis is assumed to influence the way in which individuals deal with life events and thus with stressors to which they are exposed [29]. Empirical studies found a significant association between adverse life events encountered during development [30-31] as well as adulthood [32-34] and increased diathesis for depression. Major adverse life events related to depression seem to involve experiences of threat, loss, and humiliation [32, 35]. Therefore, changes in behavior that occur as a result of such experiences, i.e. learning to cope with negative events, may become central for the understanding of depression.

3. Associative learning and depression

About a century ago, Thorndike [36] proposed that learning reinforces the formation of connections or associations between stimuli and responses, whenever a response is followed by a positive outcome (law of effect). In parallel, Pavlov [37] found that repeated pairings of a neutral stimulus (e.g., a ringing bell) with an unconditioned stimulus (e.g., food pellets) qualify the neutral stimulus to trigger (almost) identical physiological reactions as the unconditioned stimulus. That is, the unconditioned reaction (in this case: salivation), which was initially only released by the unconditioned stimulus, came elicited by the neutral stimulus. In conditioning terminology: The neutral stimulus became a conditioned stimulus triggering a conditioned reaction (for reviews, see [38-39]). Consequently, Pavlovian condition is traditionally conceptualized as learning through “stimulus substitution”. The influential Rescorla-Wagner model of conditioning, however, rejects the classical notion on how Pavlovian conditioning is working: “Pavlovian conditioning is not the shifting of a response from one stimulus to another. Instead, conditioning involves the learning of relations among events that are complexly represented, a learning that can be exhibited in various ways” [40]. Thus, modern Pavlovian thinking highlights the information that one stimulus gives about another and that organisms adjust their Pavlovian associations for their internal representation of the world. This implies that associative learning advances only to the extent to which a reinforcer is unpredicted (in terms of producing a prediction error) and slows progressively as the reinforcer becomes more predicted. Therefore, learning is assumed to be driven by changes in the expectations about salient events such as rewards and punishments [41].

Dysfunctional associative learning in terms of both instrumental (operant, Thorndikean) and respondent (Pavlovian) conditioning has been related to the development and maintenance of depression. Most remarkably, altered associative learning seems particularly linked to enhanced sensitivity for negative events and impaired responsiveness to positive stimuli in depression [42-43].

3.1. Altered aversive instrumental conditioning and learned helplessness in depression

Learning from the consequences of one’s own behavior is central to instrumental conditioning. Against the background of cognitive theories proposing that depression is associated with negative attitudes and assumptions, depressed individuals are suggested to show increased sensitivity for negative outcomes and feedback. In addition to depressed mood, anhedonia is one of the core symptoms of depression and depict the loss of interest in originally rewarding or enjoyable activities. Thus, reduced responsiveness to positive outcomes should be evident in depression as well.

Numerous cognitive tasks have been applied to elucidate the neuropsychological profile of depression [10]. Some of these tasks provide direct information about performance accuracy and depressed individuals have been found to show biased responding to negative feedback in terms of a “catastrophic response to perceived failure” [44]: When depressed individuals make a mistake, their subsequent performance deteriorated considerably. In addition, depressed individuals showed such impairment when objected to false negative feedback in tasks known to be dependent on the integrity of the neural affective loop circuitry [45-46]. It has been concluded that failure feedback can exert its influence on cognitive performance by altering the attentive focus toward increased negative focussing on the self, and that this attentional shift might decrease the cognitive resources available for the task [45]. These findings suggest that depressed individuals are in particular vulnerable to negative feedback, what might constitute a major etiological factor for the disorder. However, the question why depressed individuals show altered responding to errors and negative feedback can not be answered by means of neuropsychological tests. To this end, experimental paradigms which manipulate psychological variables related to negative events are mandatory. This was done in paradigms investigating learned helplessness.

Incidentally found in animals [47-48], learned helplessness gives an explanation for the observation that exposure to inescapable aversive events leads to a subsequent deficit in escape or avoidance behavior. Mirroring instrumental learning theory, which proposes that subjects learn that their behavior controls reinforcement, learned helplessness proposes that subjects learn that their behavior cannot control reinforcement [49]. In contrast to animal research, however, few studies have used the original (triadic) experimental design to investigate learned helplessness in humans [50]. Moreover, most of these studies did not assess the neural correlates of learned helpless behaviors and they were often conducted in healthy individuals or analogue subjects rather than clinically depressed patients.

3.1.1. From learned helplessness to hopelessness depression

To evaluate learned helplessness findings in humans, it is important to bear in mind the methodological procedure of the original animal paradigm (cf. [51]). In the original protocol, animals were first subjected to aversive respondent conditioning, in which a light was repeatedly paired with electric footshocks, while the animals were restrained in a Pavlovian harness. Subsequently the majority of animals (but not all) failed to learn to escape or avoid footshocks in a shuttlebox. Further experimental variations found the light unnecessary for this effect, and evidenced the inescapable and unavoidable shocks as the causative agent. Groups exposed to unescapable and unavoidable shocks versus escapable and yoked inescapable shocks have been compared. To this end, shocks were applied at the same time (frequency) and for the same intensity in the animals that could escape and their yoked partners. The shocks terminated when the animals which had the possibility to escape made an instrumental response (i.e., hitting a panel). Importantly, hitting the panel had no consequences in the yoked animal group: Aversive shocks were inescapable, and only animals receiving yoked inescapable shocks showed a subsequent learning deficit. Thus, uncontrollability over the aversive shocks was proposed as key variable in producing later failure to learn and consequently termed as learned helplessness (for a critical discussion on the term learned helplessness versus interference, see [52]).

According to these experimental results, two fundamental components are essential for the investigation of learned helplessness: First, it is crucial to show that indeed uncontrollability over aversive events is the driving force of learned helplessness. This implies a comparison between conditions in which subjects were exposed to uncontrollable, relative to exactly equal controllable stressors. Second, in the original procedure, subjects received uncontrollable stressors in a different arrangement (Pavlovian harness) than the one which was used to test for learned helplessness effects (shuttlebox). Therefore, learned helplessness includes trans-situationality as part of the original definition [51]. Moreover, the fully established triadic design includes a control group, which is naïve to aversive stimulation. That is, both the escape and yoked group have identical aversive stimulation as compared to the naïve group, but uncontrollability over aversive electrical stimulation is only present in the yoked group. As a well replicated finding, the naïve group shows a comparable level of escape behavior as the escape group and thus learned helplessness effects can be attributed to the loss of control over aversive events in the yoked group (cf. [53]).

Possible consequences of stressor uncontrollability range from cognitive, motivational, and emotional alterations [54] to neuroendocrinological as well as functional and structural brain changes [55] that are in line with core features of depression. However, it is noteworthy that learned helplessness was initially not conceptualized to provide an animal model of depression or any other psychopathological condition [51]. Nevertheless, there is an obvious analogy to emotional, motivational, and cognitive complains of depressed individuals: Increased negative emotions, reduced motivation, and reduced cognitive abilities to establish adaptive behavior to cope with stressors. Thus, it is reasonable to assume that if individuals learn that their behavior cannot control aversive events, negative emotionality becomes persistent and the motivation to actively manipulate stressful situations decreases in line with reduced awareness for potentially changed contingencies between own behavioral responses and environmental events. In addition, based on the trans-situational nature of the learned helplessness paradigm, learned helpless behavior is assumed to generalize across contexts with also future events being expected as uncontrollable.

Experimental studies in healthy humans widely validated learned helplessness results from animal research. By translating the general experimental procedure into human laboratory protocols, several aversive stimuli have been employed, such as electric and heat shocks, loud noise, and challenging cognitive tasks. In such a way learned helplessness effects were demonstrated for reduced escape behavior to aversive stimuli and reduced performance on cognitive tasks in healthy humans (for a review, see [56]). However, studies which focussed on the generalization of learned helplessness did not always show unambiguous results [57]. Moreover, it remains an open question as to which extent these experimental findings can be transferred to real-life settings. In addition to the assumption that repeated exposure to uncontrollable aversive events might increase generalization of learned helpless behavior, it has been proposed that generalized learned helplessness is dependent on the strength of aversive outcomes. That is, generalized learned helplessness is more likely to occur when the outcome is highly aversive, or when a highly desired outcome is not reachable by the individual [56]. Uncontrollable adverse life events, such as loss and humiliation might have the potential to induce long-lasting learned helplessness effects. At least such life events have been found to predict depressive episodes [58]. However, independent of whether negative events are objectively controllable or not, the manner of how individuals attribute the causes of negative events seems essential. This cognitive aspect was addressed in the revised learned helplessness theory [56]. Based on social attribution theory [59], revised learned helplessness theory proposes that individuals attribute causes on several dimensions: internal/external, stable/unstable, and global/specific. Hence, highly internal, stable, and global attributions for negative outcomes would relate to low self-esteem and helplessness depression. Moreover, a subsequent reformulation – the hopelessness theory of depression – suggests that latent attributional diatheses combined with stressors produce a specific subtype of depression, i.e. hopelessness depression [8, 60]. This subtype of depression is characterized by dispositional negative expectations that desired outcomes will never occur and that one’s own behavior is not effective for realizing desired outcomes (hopelessness).

Taken together, original learned helplessness theory proposed uncontrollability over aversive events, which in conditioning terminology depicts noncontingency between behavioral responses and reinforcement, as key variable for subsequent deficits in instrumental learning. The learned helplessness effect involves emotional, motivational, and cognitive characteristics obviously mirroring constituent parts of depressive symptomatology. Refined learned helplessness theory subsequently focussed on negative self-referential attributional style as a prerequisite for depressogenic behavior. Finally, the hopelessness theory of depression proposed a subtype of depression to be fundamentally related to habitual negative expectations about the self and outcomes (hopelessness). Thus, bringing the original assumptions of learned helplessness to clinical depression provoked a change in meaning for the causal importance of uncontrollability. In contrast to the original finding that uncontrollability over aversive events results in depression-like emotional, motivational, and cognitive alterations, the hopelessness theory of depression treats learned helplessness (caused by uncontrollability) not as a cause but as a necessary, however, not sufficient component of generalized hopelessness. Beside other critical issues, this conceptual development was mainly driven by the question whether or not learned helplessness does generalize across contexts in humans. Both revised learned helplessness and hopelessness theory suggest additional cognitive variables (causal attribution, negative inferential style) to be necessary for generalization.

Taking account of cognitive variables for the understanding of depression is beyond dispute. However, proposed effects of these variables have been obtained mainly by means of psychometric questionnaires which measure, e.g., inferential and response style [61] and hopelessness [62]. In addition to this approach and in the context of learned helplessness theory, it is desirable to have more direct data on cognitive mechanisms and related brain functioning when individuals are confronted with aversive events. Surprisingly few studies have addressed this topic.

3.1.2. Neural correlates of uncontrollability over aversive events in humans

Alterations in neural activation related to uncontrollability over aversive stimulation have been investigated by means of electroencephalography (EEG), functional magnetic resonance imaging (fMRI), and positron emission tomography (PET).

Seminal EEG studies [63-64] used a change from an escape to an uncontrollability paradigm in healthy individuals to assess variations in slow cortical potentials (SCP) and especially the post-imperative negative variation (PINV) related to controllability versus uncontrollability over aversive stimuli. In an S1-S2 reaction paradigm, participants received two different warning stimuli that signaled either a neutral tone or an aversive noise. Participants could avoid the aversive noise by a motor response in the first half of the experiment. Control was withdrawn in the second half of the experiment without prior warning and subjects unexpectedly could no longer avoid aversive stimulation. The main finding was an increase of the PINV over frontal recording sites during the uncontrollability condition independent of the amount of aversive stimulation per se. Subsequent studies confirmed increased PINV magnitudes to be sensitively related to unpredictable changes in response outcome contingencies [65] in support of the notion that the PINV reflects contingency reappraisal of formerly learned response outcome associations [64]. However, one major methodological aspect distinguishes paradigms for the investigation of the PINV as an electrophysiological index of altered information processing related to loss of control from paradigms used to investigate learned helplessness: Original PINV paradigms started with a condition in which subjects were able to control aversive stimulation followed by a condition of loss of control. This is exactly in reversed order as compared to learned helplessness studies. Against this background, a recent EEG study [66] expanded the traditional PINV paradigm. In this study, a modified forewarned (S1-S2) reaction paradigm was presented to three groups. The S1 always signaled subjects to prepare for the imperative stimulus (S2). In case of aversive stimulation, a short electrical shock was applied to the index finger of the non-dominant hand following S2. During blocks of control, correct button presses to the imperative stimulus avoided electrical stimulation. During uncontrollability, participants received electrical stimuli in randomized order in half of the trials irrespective of their behavioral responses (response outcome noncontingency). One group started with a waiting block followed by a block of uncontrollability and a final block of control. The authors called this group the learned helplessness group, since active conditions started with uncontrollability followed by controllability. The experience of uncontrollability was assumed to result in enhanced PINV magnitudes in the following condition of control. As a learned helplessness effect enhanced PINV magnitudes should reflect enhanced response outcome contingency processing (ambiguity) in a condition where aversive stimulation is objectively controllable. For validation, a second group was introduced, which also started with a waiting block, however, followed by two successive blocks of control. In contrast to the learned helplessness group, this group received constant control and was expected not to show PINV alterations during the final block of control. In addition, the study investigated a third group, which initially received a block of control, followed by a block of loss of control and a final block of restitution of control. This group was assumed to show immunization against learned helplessness, as human [67] and animal studies [68] found that initially experienced escapable shocks which were followed by inescapable shocks do diminish learned helplessness effects. Compared to the constant control group, the learned helplessness as well as the immunization group showed enhanced frontal PINV magnitudes during the second block (uncontrollability). This finding indicates that prior contingency learning (immunization group) does not affect the immediate impact of stressor uncontrollability. However, during the final block where all groups were able to control aversive electrical stimulation, only the learned helplessness group showed enhanced PINV magnitudes. These results are in line with the assumption that uncontrollability over aversive events alters subsequent instrumental learning when control is reestablished. Moreover, the experience of control prior to loss of control seems to protect against biased information processing during restitution of control.

These findings expand on previous results for SCP changes in healthy individuals during blocks of solvable (control) followed by blocks of unsolvable (loss of control) items of a reasoning task [69-70]. Low resolution electromagnetic tomography (LORETA; [71]) of these data found activation in Brodmann area (BA) 24 in the anterior cingulate cortex (ACC) significantly associated with the processing of uncontrollability. Source localization analysis of the PINV by means of sLORETA [72] also identified BA 24 in the anterior cingulate cortex as a core region for PINV generation [73] (see Figure 1). A recent review [74] of neuroimaging studies for this region suggest that negative affect, pain, and cognitive control is processed in this area, which is located in the anterior midcingulate cortex (aMCC). The aMCC is proposed to constitute a central relay or hub node which links information about reinforcers to motor centres responsible for expressing affect and executing aversively motivated instrumental behaviors. In the context of uncontrollability over aversive stimuli, a fMRI study also found increased activity in several brain regions (secondary somatosensory cortex and insula) including the ACC when a heat pain stimulus was perceived as uncontrollable [75]. Therefore the ACC and especially the aMCC might represent a cardinal region for the processing of instrumental contingencies related to (un)controllability over aversive events. However, a PET study [76] did not find alterations in ACC activity linked to the processing of solvable versus unsolvable items in a reasoning task. This study discovered increases in regional cerebral blood flow in the hippocampus and decreases in the mammillary bodies during solvable items. Subsequent unsolvable items were associated with decreases in hippocampal regions and increases in the mammillary bodies and the amygdalae. Therefore and in addition to the proposed key role of the aMCC, subcortical limbic areas in concert with other frontal, temporal and parietal areas seem to be engaged in resolving instrumental conflicts during uncontrollability over aversive stimulation [73].

Figure 1.

a) Averaged slow cortical potential (SCP) showing the post-imperative negative variation (PINV) in the post-S2 interval during a S1 (warning stimulus) S2 (imperative stimulus) paradigm used in [66,77-79], R: reaction (button press), ES: (potential) electrical stimulation; (b) Source localization analysis of the PINV showed BA 24 in the anterior midcingulate cortex as key region for PINV generation [73] (the centre of mass location (5,5,30) from [73] is shown as a red sphere on the standard Colin brain).

3.1.3. Neural correlates of uncontrollability over aversive events in depressed individuals

Neural correlates of altered instrumental learning related to learned helplessness in depressed individuals have been investigated solely by means of EEG studies which focussed on the PINV. In comparison to healthy individuals both anhedonic individuals [80] and depressed patients [81] have been found to show enhanced PINV magnitudes when aversive stimulation was uncontrollable or when control was restricted. These findings suggest that depression is associated with increased vulnerability to uncontrollable aversive events. By using the modified S1-S2 reaction paradigm, which includes successive conditions of control, loss of control, and restitution of control, it was demonstrated that depressed individuals show enhanced frontal PINV magnitudes during both loss of control and restitution of control [77]. Most remarkably and consistent with learned helplessness theory, the experience of uncontrollability seems to bias subsequent cortical processing in depressed individuals during a condition, where control over aversive stimulation is objectively reestablished. In addition, depressed individuals in this study showed enhanced ratings of helplessness in the restitution of control condition. Moreover, increased habitual symptom-focused and self-focused rumination were significantly linked to frontal PINV magnitudes during restitution of control in depressed individuals. For the first time, these results suggest a substantial relation between cognitive vulnerability markers of depression (rumination) and altered psychophysiological functioning during instrumental learning in depressed individuals. These results were confirmed in a follow-up study (T2) taking place six months after the initial assessment (T1) [79]. Alterations in PINV magnitudes were related to concurrent depression levels in patients, and when controlling for depression severity group differences in PINV magnitudes diminished. The authors concluded that PINV alterations wax and wane in parallel to the extent of depression severity. As frontal PINV magnitudes at T1 were not predictive for the amount of depressive symptoms or diagnostic status at T2 when baseline symptom levels were controlled, it was concluded that PINV alterations in depression represent a state rather than a trait marker of the disorder.

In summary, these findings clearly indicate that depressed subjects are especially vulnerable to perceived uncontrollability over aversive events and that it is reasonable to speculate that brain regions found in healthy subjects being related to the processing of uncontrollability show altered functioning in depression. Evidence converge that the aMCC is substantially involved in the processing of stressor uncontrollability and its consequences, highlighting this region as relevant for both learned helplessness effects and the state of helplessness in depression.

3.2. Appetitive (respondent and operant) conditioning in healthy and depressed individuals

Besides enhanced susceptibility to uncontrollable negative events, depression is typically characterized by marked anhedonia. Behavioral models emphasize that loss over environmental reinforcement is linked to reduced reward-related behavior in depression [82]. A deficient instrumental response to appetitive contingencies has also been proposed by animal models of depression [83]. In humans, anhedonia seems to be linked to dysfunctions in mesocorticolimbic dopaminergic projections from the ventral tegmental area to the dorsal and ventral striatum (including the nucleus accumbens), amygdala and hippocampus, anterior cingulate (including the subgenual portion), and ventral prefrontal cortex; circuits known to be related to the processing of reward [84-85]. In addition, the orbitofrontal cortex is involved in reward-related decision processes [86]. While functional abnormalities in these regions have been identified in depressed patients [87-90], few studies have examined specific deficits in reward processing in depression and underlying neural alterations.

In neuropsychological tests, depressed individuals show delayed responses to positive stimuli in affective signal detection tasks and a reduced positive attentional bias during facial expression identification [91-94]. In a fMRI study with medicated depressed patients, anhedonia was found to be linked to increased activation in the ventromedial prefrontal cortex and to reduced striatal activity in response to happy faces, suggesting that prefrontal activation might compensate for reduced striatal activation [95]. Healthy but not depressed individuals showed bilaterally increased activity in the fusiform gyrus and the right putamen to expressions of increasing happiness, while depressed individuals showed increased activity in the left putamen, left parahippocampal, right fusiform gyrus and amygdala to expressions of increasing sadness [96]. Another fMRI study used a dopaminergic probe to directly stimulate the human reward system [97]. Depressed individuals showed hypersensitive behavioral responses to the rewarding effects of d-amphetamine in line with altered brain activity in the ventrolateral prefrontal cortex, the orbitofrontal cortex, the caudate, and the putamen.

As noted above, modern theories of associative learning emphasize the fundamental role of predictions (and surprise) in both Pavlovian and instrumental conditioning [39, 86]. The prediction error denotes the discrepancy between a received reinforcer and its prediction. Learning is proportional to the prediction error and reaches its asymptote when the prediction error approaches zero after several learning trials. In humans, a number of fMRI studies (cf. [98]) have investigated reward-prediction. By means of probabilistic tasks in which individuals learn to make a choice that gives monetary gains or avoids losses, it has been found that short-term reward prediction is positively correlated with activation in the caudate and ventral striatum and the lateral orbito-frontal cortex, while longer-term reward prediction is positively correlated with activation in the dorsolateral prefrontal cortex and the inferior parietal cortex. The ACC was found to be involved in monetary gambles with high versus low monetary risk. However, gain versus loss outcomes seem to activate medial frontal areas and the ventral striatum including the nucleus accumbens. A fMRI study in healthy individuals [99] found the nucleus accumbens proportionally activated to the magnitude of anticipated gains, whereas the medial prefrontal cortex showed activation changes in relation to the probability of anticipated gains. Similarly, activation of striatal regions has been found to reflect differences in magnitude and probability of reward and also medial prefrontal cortex activation seem to vary with the probability of reward [100]. In addition, it was shown that activation in the caudate and ventral striatum is positively correlated with behavioral indices of reward learning and that the caudate displays increased activation in early stages of learning. Moreover, it was shown that activation in the ventral striatum is positively correlated with prediction error signals during both Pavlovian and instrumental conditioning [101]. Furthermore, the ventral striatum was found to respond to a conditioned stimulus which predicts reward delivery and seems to be characterized by a strong outcome-related response when reward is delivered unexpectedly or a decrease in activity when an expected reward is omitted. In addition, linear increases of activation were observed in the nucleus accumbens with increasing reward probabilities [102].

Yacubian and colleagues [103] were the first to use a gambling task with different gain and loss magnitudes and probabilities. Only gain-related predictions and associated prediction errors were found to be expressed in the ventral striatum, while loss-related predictions and related prediction errors were localized in the amygdala. Therefore, the authors proposed two dissociable value systems for gains and losses and suggested that the ventral striatum generates value predictions to which actual outcomes were compared, while the amygdala predicts possible losses and compares these predictions against actual outcomes.

Recent studies in depression research used variations of the monetary incentive delay paradigm to investigate neural similarities and disparities between the anticipation and receipt of reward and punishment [24, 104]. In this paradigm, trials start with the presentation of a cue which indicates a potential outcome (win/loss) followed by an imperative stimulus to which subjects have to respond with a button press. After the motor response subjects receive feedback about their actual outcome (win/loss). This protocol allows differentiating between anticipatory neural responses (in the time interval between the presentation of the cue and the imperative stimulus) and neural responses related to the presentation of the outcome (win/loss feedback). During anticipation motivational processes (wanting) are assumed to be linked to outcome-predicting cues, whereas during the outcome phase emotional responses (liking) may dominate neural responsivity, in turn providing reinforcers to foster learning about the relationship between cues and outcomes. Depressed individuals have been found to show reduced activity in fronto-striatal regions during both reward anticipation [105-106] and outcome [24, 104], suggesting that dysfunctional incentive processing in MDD is particularly linked to functional alterations in fronto-striatal regions. However, neuroimaging studies have also found intact responsivity in the ventral striatum including the nucleus accumbens [24] and enhanced anterior cingulate cortex activity [104] during reward anticipation. In addition, increased frontal activity but reduced activity in the caudate was found during the anticipation and receipt of reward in medication-free depressed adolescents [107]. Euthymic patients were found not to show fronto-striatal hypoactivity during the anticipation and receipt of reward [108]. Moreover, only few studies have investigated reward-related prediction error signaling in depression. Their results are equivocal with studies showing enhanced activity in prefrontal, striatal and ventral tegmental areas coding reward-associated prediction errors [109-111] but also reduced prediction error signaling over time in the ventral striatum and the dorsal ACC in depressed individuals [110].

Therefore and although reduced reward processing in depression seem substantially associated with anhedonia, the neural signature of reduced reward responsiveness in MDD is still a puzzling topic. Direct and indirect evidence point to fronto-striatal regions to be substantially involved during both the expectation and receipt of positive outcomes. However, further studies are clearly needed to validate these findings. Despite differences in sample characteristics, future studies may additionally focus on the interaction of brain regions involved in altered reward processing in MDD to further elucidate the current heterogeneity of findings.

4. Conclusions

In summary, recent neuroimaging results clearly demonstrate (a) increased sensitivity of depressed individuals to loss of control during instrumental conditioning, subsequently (b) causing biased information processing of actually controllable aversive events. This dysfunctional learning mechanism, which is (c) linked to negative self-referential cognition (rumination), represents (d) a valid state marker of the disorder. Brain regions of interest for altered instrumental learning in MDD seem to include (e) the anterior cingulate (in particular the aMCC), prefrontal regions, and limbic structures (amygdala, hippocampus). From a respondent perspective, alterations in associative learning mechanisms were evident (f) already during the anticipation of positive (rewarding) outcomes, most probably associated (g) with reduced prefrontal, striatal, and limbic activation for positive outcomes and (h) altered prediction error signaling in the ventral striatum and the ACC. However and as mentioned above, the latter findings need further validation in future neuroimaging studies.

Cognitive models of depression substantially benefit from current findings on altered associative learning mechanisms in MDD. Clinical interventions based on cognitive models such as cognitive behavioral therapy emphasize cognitive restructuring and behavioral activation for the treatment of depression. Increasing knowledge about the psychophysiological correlates of altered associative learning in MDD may result in focussing on interventions helping individuals suffering from MDD to experience controllability and hedonia. Future pre/post treatment studies should make use of neuroimaging methods to demonstrate treatment-specific effects of such tailored interventions on the neurobiological level. Moreover, recent approaches with neurofeedback showed the feasibility of brain self-regulation to upregulate brain areas involved in the generation of positive emotions in depressed patients [112]. Neurofeedback as a holistic approach that overcomes bio-psychological dualisms has fascinating advantages especially in the case of depression: By the use of operant learning mechanisms patients experience successful self-regulation of their own brain activity.

Acknowledgement

This work was supported by the Deutsche Forschungsgemeinschaft (SFB 636/D4). The author gratefully acknowledges the contribution of Christine Kuehner, Bettina Ubl, and Herta Flor and the valuable assistance of Veronika Thomas for the preparation of this chapter.