This chapter discusses some approaches to computational modelling of decision making. Concretely, it concerns with connectionist models of decision making and it contributes to the categorization of such models. The models presented in this chapter are algorithmic and structural descriptions of the mental process of decision making.
First of all, there are some definitions that must be stated in this chapter. A decision occurs when a person faces several options (alternatives), then evaluates the consequences of choosing each alternative and, finally, selects one depending on her/his preferences (Rustichini, 2009). Preference is an abstract relation between two options: when presented with options A 1 and A 2 , it is assumed that a subject either prefers A 1 to A 2 or the subject prefers A 2 to A 1 (or is indifferent between them). The decision is guided by the subjective experience and preference of the subject. It depends on internal factors of the subject and external factors of the environment. Due to these considerations, the goodness of a decision is subjective and it should be considered only within the context of the subject and the environment. The parameters that characterized each alternative are called criteria.
A model is a simpler and more abstract version of a system that keeps its key features while omitting less relevant details. It is used to investigate a system or phenomenon that is too complex to deal with directly. An important class of models is represented by computational models (Fum et al., 2007), which are implemented as algorithms. While statistical and mathematical models describe a phenomenon without reproducing it, computational models do. Therefore, computational models make easier the observation and measurement of a phenomenon’s behaviour.
There are two opposite points of view concerning how the human mind works (Chown, 2004). One considers the basis of human mind as a symbol processing system and the other assumes that the brain must be modelled in neural terms. This chapter is focused on connectionism, which is a theoretical framework for cognition that assumes two main statements. The first one is that cognitive phenomena arise from the propagation of activation among nodes. The second is that such propagation is mediated by weighted connections between nodes. So, computational models built on connectionism principles are composed by nodes and connections. A node represents an entity (idea, concept, etc.) which has an associated value (activation). A node can transmit its activation through its connections. One node can either excite or inhibit another node depending on the strength of the connection that links them. Thus, a connectionist model must specify, among other things, the way nodes transmit activation.
There are neuropsychological evidences that suggest that the human mechanism for making a decision is divided into two stages (Glimcher, 2009). The first stage lies in the assessment of all alternatives and the second is concerned with choosing one of them depending of the value assigned in the previous stage. The assessment mechanism is associated with the representation of values, while the choice mechanism is associated with a process that takes these values and returns the best alternative. These mechanisms are the core of the models presented in this chapter. These evidences confirm the Prospect theory (Kahneman & Tversky, 1979), which distinguishes two phases in the choice process: an early phase of editing and a subsequent phase of evaluation. The editing phase consists of a preliminary analysis of the offered options, which often yields a simpler representation of these options. In the second phase, the edited options are evaluated and the option of highest value is chosen.
There are several models that describe the decision process. Depending on the application, different constraints may be enforced on the computational modelling task. This chapter deals with the following categories of models. Threshold Models make a decision when there is sufficient evidence accumulated favouring one alternative over the others. Ranking models rank the alternatives according to their estimated consequences and then choose the best one. Rule-based models apply rules to choose the best alternative. Emotional models take into account emotion in the decision process. Physiologically motivated models aim to describe the decision process using interconnected modules which represent different brain areas. An important remark is that some of the models presented in this chapter fall into more than one category.
This chapter is organized as follows. Section 2 presents the category of Threshold Models and explains different models within this category. In section 3 there are some Ranking models. Section 4 describes Rule-based models. Section 5 contains different models that include emotions in the process of decision making. Some physiologically motivated models are compiled in section 6. Finally, section 7 presents the conclusions.
2. Threshold Models
Threshold Models assume that decisions are based on accumulated information about the alternatives. Therefore, a decision is the result of continuously accumulating information until a threshold is reached. Threshold Models emulate the decision process as a race between alternatives, with the response determined by the first alternative to reach a threshold. The threshold that determines the amount of information needed for a response is under the control of the needs of the decision maker (the subject that makes the decision). For instance, the threshold is reduced with the necessity to respond quickly and increased with the necessity to respond accurately. Two main features of these models are the starting value of the accumulation process and the thresholds. The interest of these models is that they provide a description of the relationship between time and accuracy, and hence they are suitable for modelling speed-accuracy decision effects. In the following sections different Threshold Models are discussed.
Within the category of Threshold Models there are differences in how the accumulation is assumed to occur. These models contain a node, which is called accumulator, for each alternative, i.e. the information favouring each alternative is accumulated separately. Threshold Models gather information through other kind of nodes that represents environmental and subject’s features. In Dependent Accumulators models, information in favour of one alternative is the evidence against the others. Thus, the accumulators are mutually inhibitory. There is another class in which accumulators are independent and there is no inhibition: Independent Accumulators models.
2.1. Decision field theory
Decision Field Theory (Busemeyer & Townsend, 1993) is a dynamic model of decision making that has been used to explain different aspects of the decision process such as the similarity effect, the attraction effect, the compromise effect and preference reversals (Roe et al., 2001; Johnson & Busemeyer, 2005). This model assumes that the decision process can be formulated as a connectionist model as shown in figure 1.
Information about the various possible consequences of each alternative represents the inputs into the model (C i ), i.e. criteria. The input criteria are filtered by an attention process, which maps the evaluations of consequences into a momentary evaluation for each alternative, represented by the second layer of nodes. Then the momentary evaluations are transformed into valences, one for each alternative, represented by the third layer of nodes. The valence of an alternative represents the momentary advantage or disadvantage of that alternative compared to the other alternatives. Finally, the valences are input to a recursive system at the final layer which generates the accumulation of information favouring each alternative at each moment in time. Decision Field Theory calls preferences to the values accumulated in the last layer. In this model, as attention switches across criteria over time, the accumulation of information also varies until the preference for one alternative exceeds the threshold and the winner is chosen.
The assessment mechanism lies in the first layer where information is gathered and criteria are weighted. The choice mechanism lies in the accumulation process through the last three layers of the model. This model is an example of a Dependent Accumulators model due to the connections between accumulators in the last layer. Notice that there exists inhibition.
2.2. Leaky, competing accumulators
The model presented in this section is based on the Leaky, Competing Accumulator model (Usher & McClelland, 2001) and operates as follows. At each time step, one criterion (C i ) of the consequences of the alternatives is chosen randomly to be the focus of attention. Therefore, the selected criterion is the only one that transmits its activation. The input to each of the Leaky, Competing Accumulator nodes (accumulators) is determined by an input pre-processing stage. This stage calculates the differences (dij) between all pairs of alternatives over the chosen criterion and then, converts these differences before transmitting them to the accumulators. This stage is performed in the second and third layer. The nodes in the second layer represent each alternative according to its weights over the criteria and transmit their activation to the third layer. The nodes in the third layer compute and transform the differences between the alternatives and, finally, transmit them to the last layer, which contains the accumulators.
Figure 2 illustrates this model for a situation with three alternatives and two criteria. The model presented in this section (Usher & McClelland, 2004) extended the Leaky, Competing Accumulator model of perceptual choice incorporating switching of attention between criteria.
As in the Decision Field Theory, the assessment mechanism lies in the first layer and the choice mechanism consists in accumulating information through the different layers of the model. This model is another instance of a Dependent Accumulators model.
2.3. Accumulator Model
The accumulator Model (Vickers 1970; Smith & Vickers, 1988) deals with decisions that have two alternatives. In this model there is no inhibition between the two accumulators. Each criterion (C i ) characterizing the alternatives (Ai) has associated a reference value (Ri). At each time step, one criterion is selected randomly and if its value is greater than the reference value, then the model adds the difference between the reference and the value to one accumulator. If the value is lower than the reference, then the model adds the difference to the other accumulator. The decision is determined by the first accumulator to reach the threshold. Figure 3 shows a connectionist interpretation of this model that contains two accumulators, one for each alternative, and two criteria.
In this model, the choice mechanism is represented in the accumulation process and the assessment mechanism is based on the representation of the criteria. Notice that there is no inhibition between accumulators.
3. Ranking models
This approach makes the assumption that there is a global comparison of the alternatives. These models lie in the evaluation of the alternatives over each criterion and the determination of a score for each alternative. After this process of assessment, they determine a global ranking on the alternatives based on the scores. The decision is stated by the alternative with the best score. One of the most difficult tasks is to normalize the original values of the criteria, i.e. the assessment mechanism.
Within the category of Ranking models there are differences in how the global score is computed. For instance, a model built on these principles can lie in computing the weighted sum of some partial scores given by the criteria that characterize the alternatives. This chapter presents different Ranking models in the following sections.
3.1. Fuzzy cognitive map
A fuzzy cognitive map (Kosko, 1986) is a technique for modelling complex systems that consists of a great number of highly related and interconnected elements. Fuzzy cognitive maps represent knowledge capturing the cause and effect relationships among elements in order to model the behaviour of a system. The first fuzzy cognitive maps used numbers to describe causality and introduced positive and negative concepts. Fuzzy cognitive maps have been extended in order to be applied to decision making (Montibeller & Belton, 2009).
A kind of fuzzy cognitive map developed for decision making is called competitive fuzzy cognitive map (Stylios et al., 2008). The competitive fuzzy cognitive map introduced the distinction of two main kinds of concepts: decision concepts and factor concepts. Figure 4 illustrates a competitive fuzzy cognitive map that includes three alternatives (A 1, A 2 and A 3 ) and several criteria (C 1 , C 2 and C 3 ). All the nodes can interact with each other and determine the value of the alternatives, which are mutually exclusive, in order to indicate always a single decision. The connections between concepts represent the causal relations among them. This model operates as follows. The model assigns the activation of factor nodes according to the decision making problem. These nodes are the input of the model. Then, the model begins a simulation divided in time steps. At each simulation step, the activation of a node is calculated by computing the influence of the interconnected nodes on the specific one following a sigmoid threshold function. The simulation finishes when there are no variations in the activation of every node. In such situation, when the competitive fuzzy cognitive map has converged, the decision node that has the greatest activation represents the best alternative.
In this model the assessment mechanism is represented by the propagation of activation through the nodes of the map representing criteria and the choice mechanism consists in selecting the alternative associated to the best scored decision node. Notice that there is an inhibition between alternatives. This is not a Threshold model because the decision is not made when a decision node reaches a threshold. Instead of it, the decision is made when the map has converged.
3.2. Hybrid model
The model presented in (Iglesias et al., 2008a) is composed of an artificial weighted net of concepts, an evolution module, a transformation module and a assessment module. The net of concepts represent the environment and the expert knowledge about the criteria involved in a decision. A net concept stands for a criterion (C i ) or an event (E i ) whose value may depend on the values of other different events. The association weights between net concepts, like in the rest of the presented models, are considered as a level of influence of the source concept on the destination concept.
The evolution module takes the values of the criteria stored in the net of concepts and modifies them depending on each alternative. This module predicts the environment evolution, i.e. the consequences a decision would produce over the environment and hence, over the values of the criteria. The transformation module applies a fuzzy transformation (T ij ) to obtain three discrete values (maximum, most possible and minimum value) from each evolved criterion. Finally, the assessment module takes the discrete values and scored each alternative using one of three evaluation methods: the general fuzzy method, a fuzzy method based on eigenvectors or influence diagrams. The assessment module ranks the alternatives depending on the score computed by the selected evaluation method and chooses the best alternative.
As figure 5 shows, the assessment mechanism of this model lies in the first three layers. The choice mechanism is represented by the evaluation method applied in the assessment layer.
4. Rule-based models
These models assume that multiple decision rules coexist in the brain. Some rules are based on heuristics and other rules involve deliberative calculation. On the one hand, heuristics rules allow decision makers to avoid irrelevant information and make fast decisions. On the other hand, deliberative rules allow decision makers to evaluate complex situations in order to extract relevant information.
An example of this category of models is DECIDER (Levine, 2009). This model is composed of a module that represents the decision maker’s needs and a module of decision rules. The state of the needs module determines which rule must be applied through the orienting module. Depending on the pattern identified by the network contained in each rule module, the model applies the corresponding rule. Figure 6 shows a simplified version of DECIDER.
The assessment mechanism is compiled in the needs module that establishes which rule must be applied. The choice mechanism lies in the rule that chooses the best alternative.
5. Emotional models
Neurophysiological and neuropsychological evidences demonstrate that emotions are an indispensable requirement for deciding properly (Damasio, 1994; Simón, 1998; Pessoa, 2008). Therefore, it is necessary to take into account the participation of emotions in decision processes. There are several models that include emotions in the decision process in order to describe better the decision mechanisms. The following section presents a representative instance of these models.
5.1. Integrated cognitive-motivational network
This section introduces a model for integrating cognition and emotion into a single decision process (Busemeyer et al. 2007). This model is an extension of the Decision Field Theory (Busemeyer & Townsend, 1993). The momentary evaluations of the Decision Field Theory are affected here in this model by emotions. The effect of a criterion on a momentary evaluation of a consequence depends on two factors: the quality and the need for the criterion. The quality represents the amount of satisfaction that a consequence can deliver with respect to a criterion. This model assumes that a subject has an ideal reference on each criterion as well as a current level of achievement for a criterion. The need is the difference between the reference and the current level of achievement for a criterion. The need for a criterion varies across time. These two factors, the quality and the need, are combined to produce a motivational value for a criterion. Then, motivational values are associated with the corresponding decision weights to compute the momentary evaluation.
A scheme of the integrated cognitive-motivational network that shows the influence of emotions in the decision process is shown in figure 7. This model also belongs to the category of Threshold Models because the node called preferences accumulates information about each alternative as in the Decision Field Theory.
6. Physiologically motivated models
Physiologically motivated models aim to describe the decision process using several interconnected modules which represent different brain areas (e.g. orbitofrontal cortex, amygdala, etc.). There are some brain areas closely related with decision making. One of them is the amygdala, which is capable of assigning emotional meaning to environmental stimuli. The following sections describe some of those models.
6.1. Neural affective decision theory
This theory specifies brain mechanisms underlying decision making. It lies in four principles: affect, brain, assessment and framing. Affect means that decision making is a cognitive affective process dependent on emotional evaluation of alternatives. The brain principle represents that decision making is a neural process driven by coordinated interactions among several brain areas. Assessment suggests that the brain computes preferences via two distinct mechanisms for positive and negative outcomes. Framing defines that judgments and decisions vary depending on the presentation of information.
A representative model of this theory is ANDREA (Litt et al., 2008). ANDREA is divided into seven different modules that represent major brain areas that contribute to the assessment and choice mechanisms: the amygdala, orbitofrontal cortex, anterior cingulated cortex, dorsolateral prefrontal cortex, the ventral striatum, midbrain dopaminergic neurons and serotonergic neurons centered in the dorsal raphe nucleus of the brainstem. Figure 8 illustrates the connectivity scheme between the different modules.
This model describes a biological mechanism for decision making. The assessment mechanism lies in the input to the modules representing the amygdale and the orbitofrontal cortex. The choice mechanism is based on the recurrent connections between all the modules. This model is also in the category of ranking models.
Another physiologically motivated model presented in (Wagar & Thagard, 2004) is GAGE. The individual neurons in GAGE are more realistic than those used in most artificial neural network models because they exhibit the spiking behaviour found in real neurons. GAGE organizes neurons into populations related to brain areas, including the ventromedialprefrontal cortex (VMPFC), the hippocampus, the amygdala, the nucleus accumbens and the ventral tegmental area. Figure 9 illustrates a diagram of the neuronal mechanism developed in GAGE.
The node representing the ventromedial prefrontal cortex receives the inputs to the model so it contains the assessment mechanism. The choice mechanism is finally set by the nucleus accumbens, which is the node with the output of the model. This model is an instance of a ranking model.
6.3. Recurrent network model
The model detailed in (Lo & Wang, 2006) consists of three brain areas: the cortex, the superior colliculus and the basal ganglia. These brain areas are represented as neural networks. Each of the three networks contains competing neural populations for each alternative. Neural populations compete with each other through local recurrent synaptic inhibition. The cortical neurons slowly accumulate information about criteria. The neural population receiving a stronger input has a higher probability of reaching a threshold and winning the competition.
Figure 10 shows the model architecture. Neural pools in the cortical network integrate sensory information about criteria and also compete against each other. They propagate activation to both the superior colliculus and the basal ganglia. The superior colliculus provides the output that represents the winner alternative.
This model is an example of a Threshold model that is also physiologically motivated.
The categorization just presented here compiles existing connectionist models related to decision making. The identification of the best model depends on the task to which it will be applied. The evaluation of decision making models is typically conducted experimentally, rather than analytically. There are two main ways to evaluate a model experimentally. The first way lies in computing the ability of the model to take the right decision, i.e. the decision that produces the best outcome. However, there is another interesting way of evaluation which seeks to calculate the ability of the model to take the same decisions as a human being does on a well-defined task. Therefore, if the goal is to find the model that best describes the decision process of human beings, then it might be used the second kind of evaluation. This second way of evaluation is often used to validate a model, suggesting that a good fit to human performance is good evidence for the theory implemented by the model.
In this chapter there are several models that have been applied on different tasks, so it is very difficult to compare them. Threshold Models are often used in decision making problems where time and accuracy are the two most important features. A representative task of this kind is the two-alternative forced choice task (Bogacz et al., 2006). The competitive fuzzy cognitive map is used as a medical decision support system in differential diagnosis. The hybrid model presented in (Iglesias et al., 2008a) is applied in fire emergencies in order to choose the best action to mitigate a crisis. DECIDER is used to model preference reversals by consumers deciding among competing soft drinks. In (Busemeyer et al., 2007), the model is applied in a situation where a motorcyclist is driving towards an obstacle and she/he must decide what to do. ANDREA is used to predict decisions in problems where a human being has to choose between two different lotteries regarding the possible gains and loses and their probabilities. GAGE is used to simulate experimental results concerning the Iowa gambling task (Bechara et al., 1994). Finally, the model detailed in (Lo & Wang, 2006) has been applied in reaction time tasks similar to the two-alternative forced choice task. It would be very useful to present every model within the context of the same decision making problem. One possibility is the use of a simple problem like the one presented in (Iglesias et al., 2008b) that lies in choosing the best means of transport. With this decision making problem, the comparison of the models would be easier and the differences on the theories that they implemented could be more notable.
It seems that in neuroscience and psychology is growing the use of physiologically motivated models as a tool to both test and develop theories. This kind of models explicitly contains modules representing different brain areas. This feature is very suitable in imaging studies that measures brain activity such as functional magnetic resonance imaging (De Martino et al., 2006). It is frequent to find correlations between psychological measures and measures of brain activity (Kahneman, 2009). Therefore, the similarity between brain activities and the values of the model parameters can be interpreted as a clue in the validation of the model.
An important characteristic of a decision making model must be its ability to explain the decisions that it makes. None of the models presented in this chapter seeks the explanation of its decisions. A model developed by the authors of this chapter which aims to explain its decisions while making the same decisions of human beings obtained the second position in the Dynamic Stocks and Flows challenge (Lebiere et al., 2009). The model presented at the challenge is a connectionist model of decision making and it belongs to the Ranking models category. This result confirms that this connectionist model can both explain its decisions and simulate human performance. The explanations of the decisions may lead to better understanding of decision making and they will soon play an important role in the process of studying how people decide.