Why we need a neuroeconomic account of valuation

In my last post, I outlined an account of valuation. Whether mine is a good one is disputable, but the fact is that research in neuroeconomic, philosophy of mind, psychology, or any other field concerned with decision-making will sooner or later require a credible account of valuation, and value. Here is two reasons why.

First, there is a significant overlap between brain areas and processes in different valuations domains. While neuroeconomics, neuroethics and neuropolitics began to make explicit the neural mechanisms involved in these domains (Glimcher, 2003; Tancredi, 2005; Westen, 2007), attempts to cross-fertilize research are scarce. Research showed that economic, moral and political cognition involve similar brain processes. For instance, whether subjects play economic games (Rilling et al., 2002; Sanfey et al., 2003), reflect upon moral issues (Greene & Haidt, 2002; Koenigs et al., 2007) or make political judgments (Kaplan et al., 2007; Knutson et al., 2006; Westen et al., 2006), these tasks recruits principally the following evaluative mechanisms: Core affect, monitoring and control mechanisms described in my last post.

Second, even in one area--neuroeconomics--it is not clear what researchers mean when they talk about the "neural substrate of economic value". Take for instance two recent studies. Seo et al (2007) and Padoa-Schioppa (2007) both attempt to identify the brain valuation processes. The first one conclude that

A rich literature from lesion studies, functional imaging, and primate neurophysiology suggests that critical mechanisms for economic choice might take place in the orbitofrontal cortex. More specifically, recent results from single cell recordings in monkeys link OFC [Orbitofrontal Cortex] to the computation of economic value. We showed that the value representation in OFC reflects the subjective nature of economic value, and that neurons in this area encode value per se, independently of the visuo-motor contingencies of choice

The other discuss how the DLPFC contribute to decision-making:

individual neurons in the dorsolateral prefrontal cortex (DLPFC) encoded 3 different types of signals that can potentially influence the animal's future choices. First, activity modulated by the animal's previous choices might provide the eligibility trace that can be used to attribute a particular outcome to its causative action. Second, activity related to the animal's rewards in the previous trials might be used to compute an average reward rate. Finally, activity of some neurons was modulated by the computer's choices in the previous trials and may reflect the process of updating the value functions.

So how is something valuated? DLPFC or OFC ? How exactly they differ? Yes, one is about "reward" and the other "economic value", but again, since money can be rewarding and food can have an economic value (utility), it is not clear that different words refer to different processes. On top of that, there is also a huge literature on dopaminergic systems and valuation (see Montague et al, 2006; Montague, 2006) for a complete review). So some clarification is required here. I will try, in future post, do discuss these questions.

• Glimcher, P. W. (2003). Decisions, uncertainty, and the brain : the science of neuroeconomics. Cambridge, Mass. ; London: MIT Press.
• Greene, J. D., & Haidt, J. (2002). How (and where) does moral judgment work? Trends Cogn Sci, 6(12), 517-523.
• Kaplan, J. T., Freedman, J., & Iacoboni, M. (2007). Us versus them: Political attitudes and party affiliation influence neural response to faces of presidential candidates. Neuropsychologia, 45(1), 55-64.
• Knutson, K. M., Wood, J. N., Spampinato, M. V., & Grafman, J. (2006). Politics on the Brain: An fMRI Investigation. Soc Neurosci, 1(1), 25-40.
• Koenigs, M., Young, L., Adolphs, R., Tranel, D., Cushman, F., Hauser, M., et al. (2007). Damage to the prefrontal cortex increases utilitarian moral judgements. Nature, 446(7138), 908-911.
• Montague, P. R., King-Casas, B., & Cohen, J. D. (2006). Imaging valuation models in human choice. Annu Rev Neurosci, 29, 417-448.
  
• Montague, R. (2006). Why choose this book? : how we make decisions. New York: Penguin Group.
  Padoa-Schioppa, C. (2007). Orbitofrontal Cortex and the Computation of Economic Value. Ann NY Acad Sci, annals.1401.1011.
  
• Rilling, J., Gutman, D., Zeh, T., Pagnoni, G., Berns, G., & Kilts, C. (2002). A neural basis for social cooperation. Neuron, 35(2), 395-405.
• Sanfey, A. G., Rilling, J. K., Aronson, J. A., Nystrom, L. E., & Cohen, J. D. (2003). The neural basis of economic decision-making in the Ultimatum Game. Science, 300(5626), 1755-1758.
• Seo, H., Barraclough, D. J., & Lee, D. (2007). Dynamic Signals Related to Choices and Outcomes in the Dorsolateral Prefrontal Cortex. Cereb. Cortex, 17(suppl_1), i110-117
• Tancredi, L. R. (2005). Hardwired behavior : what neuroscience reveals about morality. New York: Cambridge University Press.
• Westen, D. (2007). The political brain : the role of emotion in deciding the fate of the nation. New York: PublicAffairs.
• Westen, D., Blagov, P. S., Harenski, K., Kilts, C., & Hamann, S. (2006). Neural Bases of Motivated Reasoning: An fMRI Study of Emotional Constraints on Partisan Political Judgment in the 2004 U.S. Presidential Election. J. Cogn. Neurosci., 18(11), 1947-1958.

A neuroecomic picture of valuation

Values are everywhere: ethics, politics, economics, law, for instance, all deal with values. They all involve socially situated decision-makers compelled to make evaluative judgment and act upon them. These spheres of human activity constitute three aspects of the same problem, that is, guessing how 'good' is something. This means that value 'runs' on valuation mechanisms. I suggest here a framework to understand valuation.

I define here valuation as the process by which a system maps an object, property or event X to a value space, and a valuation mechanism as the device implementing the matching between X and the value space. I do not suggest that values need to be explicitly represented as a space: by valuation space, I mean an artifact that accounts for the similarity between values by plotting each of them as a point in a multidimensional coordinate system. Color spaces, for instance, are not conscious representation of colors, but spatial depiction of color similarity along several dimensions such as hue, saturation brightness.

The simplest, and most common across the phylogenetic spectrum, value space has two dimensions: valence (positive or negative) and magnitude. Valence distinguishes between things that are liked and things that are not. Thus if X has a negative valence, it does not implies that X will be avoided, but only that it is disliked. Magnitude encodes the level of liking vs. disliking. Other dimensions might be added—temporality (whether X is located in the present, past of future), other- vs. self-regarding, excitatory vs. inhibitory, basic vs. complex, for instance—but the core of any value system is valence and magnitude, because these two parameters are required to establish rankings. To prefer Heaven to Hell, Democrats to Republicans, salad to meat, or sweet to bitter involves valence and magnitude.

Nature endowed many animals (mostly vertebrates) with fast and intuitive valuation mechanisms: emotions.[1] Although it is a truism in psychology and philosophy of mind and that there is no crisp definition of what emotions are[2], I will consider here that an emotion is any kind of neural process whose function is to attribute a valence and a magnitude to something else and whose operative mode are somatic markers. Somatic markers[3] are bodily states that ‘mark’ options as advantageous/disadvantageous, such as skin-conductance, cardiac rhythm, etc. Through learning, bodily states become linked to neural representations of the stimuli that brought these states. These neural structures may later reactivate the bodily states or a simulation of these states and thereby indicate the valence and magnitude of stimuli. These states may or may not account for many legitimate uses of the word “emotions”, but they constitute meaningful categories that could identify natural kinds[4]. In order to avoid confusion between folk-psychological and scientific categories, I will rather talk of affects and affective states, not emotions.

More than irrational passions, affectives states are phylogenetically ancient valuation mechanisms. Since Darwin,[5] many biologists, philosophers and psychologists[6] have argued that they have adaptive functions such as focusing attention and facilitating communication. As Antonio Damasio and his colleagues discovered, subjects impaired in affective processing are unable to cope with everyday tasks, such as planning meetings[7]. They lose money, family and social status. However, they were completely functional in reasoning or problem-solving tasks. Moreover, they did not felt sad for their situation, even if they perfectly understood what “sad” means, and seemed unable to learn from bad experiences. They were unable to use affect to aid in decision-making, a hypothesis that entails that in normal subjects, affect do aid in decision-making. These findings suggest that decision-making needs affect, not as a set of convenient heuristics, but as central evaluative mechanisms. Without affects, it is possible to think efficiently, but not to decide efficiently (affective areas, however, are solicited in subjects who learn to recognize logical errors[8]).

Affects, and specially the so-called ‘basic’ or ‘core’ ones such as anger, disgust, liking and fear[9] are prominent explanatory concepts in neuroeconomics. The study of valuation mechanisms reveals how the brain values certain objects (e.g. money), situations (e.g. investment, bargaining) or parameter (risk, ambiguity) of an economic nature. Three kinds of mechanisms are typically involved in neuroeconomic explanations:

1. Core affect mechanisms, such as fear (amygdala), disgust (anterior insula) and pleasure (nucleus accumbens), encode the magnitude and valence of stimuli.
2. Monitoring and integration mechanisms (ventromedial/mesial prefrontal, orbitofrontal cortex, anterior cingulate cortex) combine different values and memories of values together
3. Modulation and control mechanisms (prefrontal areas, especially the dorsolateral prefrontal cortex), modulate or even override other affect mechanisms.

Of course, there is no simple mapping between psychological functions and neural structures, but cognitive and affective neuroscience assume a dominance and a certain regularity in functions. Disgust does not reduce to insular activation, but anterior insula is significantly involved in the physiological, cognitive and behavioral expressions of disgust. There is a bit of simplification here—due to the actual state of science—but enough to do justice to our best theories of brain functioning. I will here review two cases of individual and strategic decision-making, and will show how affective mechanisms are involved in valuation[10].

In a study by Knutson et al.,[11] subjects had to choose whether or not they would purchase a product (visually presented), and then whether or not they would buy it at a certain price. While desirable products caused activation in the nucleus accumbens, activity is detected in the insula when the price is seen as exaggerated. If the price is perceived as acceptable, a lower insular activation is detected, but mesial prefrontal structures are more solicited. The activation in these areas was a reliable predictor of whether or not subjects would buy the product: prefrontal activation predicted purchasing, while insular activation predicted the decision of not purchasing. Thus purchasing decision involves a tradeoff, mediated by prefrontal areas, between the pleasure of acquiring (elicited in the nucleus accumbens) and the pain of purchasing (elicited in the insula). A chocolate box—a stimulus presented to the subjects—is located in the high-magnitude, positive-valence regions of the value space, while the same chocolate box priced at $80 is located in the high-magnitude, negative-valence regions of the space.

In the ultimatum game, a ‘proposer’ makes an offer to a ‘responder’ that can either accept or refuse the offer. The offer is a split of an amount of money. If the responder accepts, she keeps the offered amount while the proposer keeps the difference. If she rejects it, however, both players get nothing. Orthodox game theory recommends that proposers offer the smallest possible amount, while responder should accept every proposition, but all studies confirm that subjects make fair offer (about 40% of the amount) and reject unfair ones (less than 20%)[12]. Brain scans of people playing the ultimatum game indicate that unfair offers trigger, in the responders’ brain, a ‘moral disgust’: the anterior insula is more active when unfair offers are proposed,[13] and insular activation is proportional to the degree of unfairness and correlated with the decision to reject unfair offers[14]. Moreover, unfair offers are associated with greater skin conductance[15]. Visceral and insular responses occur only when the proposer is a human: a computer does not elicit those reaction. Beside the anterior insula, two other areas are recruited in ultimatum decisions: the dorsolateral prefrontal cortex (DLPFC). When there is more activity in the anterior insula than in the DLPFC, unfair offers tend to be rejected, while they tend to be accepted when DLPFC activation is greater than anterior insula.

These two experiments illustrate how neuroeconomics begin to decipher the value spaces and how valuation relies on affective mechanisms. Although human valuation is more complex than the simple valence-magnitude space, this ‘neuro-utilitarist’ framework is useful for interpreting imaging and behavioral data: for instance, we need an explanation for insular activation in purchasing and ultimatum decision, and the most simple and informative, as of today, is that it trigger a simulated disgust. More generally, it also reveals that the human value space is profoundly social: humans value fairness and reciprocity. Cooperation[16] and altruistic punishment[17] (punishing cheaters at a personal cost when the probability of future interactions is null), for instance, activate the nucleus accumbens and other pleasure-related areas. People like to cooperate and make fair offers.

Neuroeconomic experiments also indicate how value spaces can be similar across species. It is known for instance that in humans, losses[18] elicit activity in fear-related areas such as amygdala. Since capuchin monkey’s behavior also exhibit loss-aversion[19] (i.e., a greater sensitivity to losses than to equivalent gains), behavioral evidence and neural data suggests that the neural implementation of loss-aversion in primates shares common valuation mechanisms and processing. The primate—and maybe the mammal or even the vertebrate—value space locate loss in a particular region.

Notes

• [1] (Bechara & Damasio, 2005; Bechara et al., 1997; Damasio, 1994, 2003; LeDoux, 1996; Naqvi et al., 2006; Panksepp, 1998)
• [2] (Faucher & Tappolet, 2002; Griffiths, 2004; Russell, 2003)
• [3] (Bechara & Damasio, 2005; Damasio, 1994; Damasio et al., 1996)
• [4] (Griffiths, 1997)
• [5] (Darwin, 1896)
• [6] (Cosmides & Tooby, 2000; Paul Ekman, 1972; Griffiths, 1997)
• [7] (Damasio, 1994)
• [8] (Houde & Tzourio-Mazoyer, 2003)
• [9] (Berridge, 2003; P. Ekman, 1999; Griffiths, 1997; Russell, 2003; Zajonc, 1980)
• [10] The material for this part is partly drawn from (Hardy-Vallée, forthcoming)
• [11] (Knutson et al., 2007).
• [12] (Oosterbeek et al., 2004)
• [13] (Sanfey et al., 2003)
• [14] (Sanfey et al., 2003: 1756)
• [15] (van 't Wout et al., 2006)
• [16] (Rilling et al., 2002).
• [17] (de Quervain et al., 2004).
• [18] (Naqvi et al., 2006)
• [19] (Chen et al., 2006)

References

• Bechara, A., & Damasio, A. R. (2005). The somatic marker hypothesis: A neural theory of economic decision. Games and Economic Behavior, 52(2), 336.
• Bechara, A., Damasio, H., Tranel, D., & Damasio, A. R. (1997). Deciding Advantageously Before Knowing the Advantageous Strategy. Science, 275(5304), 1293-1295.
• Berridge, K. C. (2003). Pleasures of the brain. Brain and Cognition, 52(1), 106.
• Chen, M. K., Lakshminarayanan, V., & Santos, L. (2006). How Basic Are Behavioral Biases? Evidence from Capuchin Monkey Trading Behavior. Journal of Political Economy, 114(3), 517-537.
• Cosmides, L., & Tooby, J. (2000). Evolutionary psychology and the emotions. Handbook of Emotions, 2, 91-115.
• Damasio, A. R. (1994). Descartes' error : emotion, reason, and the human brain. New York: Putnam.
• Damasio, A. R. (2003). Looking for Spinoza : joy, sorrow, and the feeling brain (1st ed.). Orlando, Fla. ; London: Harcourt.
• Damasio, A. R., Damasio, H., & Christen, Y. (1996). Neurobiology of decision-making. Berlin ; New York: Springer.
• Darwin, C. (1896). The expression of the emotions in man and animals ([Authorized ed.). New York,: D. Appleton.
• de Quervain, D. J., Fischbacher, U., Treyer, V., Schellhammer, M., Schnyder, U., Buck, A., et al. (2004). The neural basis of altruistic punishment. Science, 305(5688), 1254-1258.
• Ekman, P. (1972). Emotion in the human face: guide-lines for research and an integration of findings. New York,: Pergamon Press.
• Ekman, P. (1999). Basic emotions. In T. Dalgleish & M. Power (Eds.), Handbook of Cognition and Emotion (pp. 45-60). Sussex John Wiley & Sons, Ltd.
• Faucher, L., & Tappolet, C. (2002). Fear and the focus of attention. Consciousness & emotion, 3(2), 105-144.
• Griffiths, P. E. (1997). What emotions really are : the problem of psychological categories. Chicago, Ill.: University of Chicago Press.
• Griffiths, P. E. (2004). Emotions as Natural and Normative Kinds. Philosophy of Science, 71, 901–911.
• Hardy-Vallée, B. (forthcoming). Decision-making: a neuroeconomic perspective. Philosophy Compass.
• Houde, O., & Tzourio-Mazoyer, N. (2003). Neural foundations of logical and mathematical cognition. Nature Reviews Neuroscience, 4(6), 507-514.
• Knutson, B., Rick, S., Wimmer, G. E., Prelec, D., & Loewenstein, G. (2007). Neural predictors of purchases. Neuron, 53(1), 147-156.
• LeDoux, J. E. (1996). The emotional brain : the mysterious underpinnings of emotional life. New York: Simon & Schuster.
• Naqvi, N., Shiv, B., & Bechara, A. (2006). The Role of Emotion in Decision Making: A Cognitive Neuroscience Perspective. Current Directions in Psychological Science, 15(5), 260-264.
• Oosterbeek, H., S., R., & van de Kuilen, G. (2004). Differences in Ultimatum Game Experiments: Evidence from a Meta-Analysis. Experimental Economics 7, 171-188.
• Panksepp, J. (1998). Affective neuroscience : the foundations of human and animal emotions. New York: Oxford University Press.
• Rilling, J. K., Gutman, D. A., Zeh, T. R., Pagnoni, G., Berns, G. S., & Kilts, C. D. (2002). A Neural Basis for Social Cooperation. Neuron, 35(2), 395-405.
• Russell, J. A. (2003). Core affect and the psychological construction of emotion. Psychol Rev, 110(1), 145-172.
• Sanfey, A. G., Rilling, J. K., Aronson, J. A., Nystrom, L. E., & Cohen, J. D. (2003). The neural basis of economic decision-making in the Ultimatum Game. Science, 300(5626), 1755-1758.
• van 't Wout, M., Kahn, R. S., Sanfey, A. G., & Aleman, A. (2006). Affective state and decision-making in the Ultimatum Game. Exp Brain Res, 169(4), 564-568.
• Zajonc, R. B. (1980). Feeling and thinking: Preferences need no inferences. American Psychologist, 35(2), 151-175.
• 
  

Beautiful picture of brain areas involved in decision-making

Found yesterday, in a paper by Sanfey (nice review paper, by the way):

"Fig. 2. Map of brain areas commonly found to be activated in decision-making studies. The sagittal section (A) shows the location of the anterior cingulate cortex (ACC), medial prefrontal cortex (MPFC), orbitofrontal cortex (OFC), nucleus accumbens (NA), and substantia nigra (SN). The lateral view (B) shows the location of the dorsolateral prefrontal cortex (DLPFC) and lateral intraparietal area (LIP). The axial section (C; cut along the white line in A and B) shows the location of the insula (INS) and basal ganglia (BG)."

from:

• Sanfey, A. G. (2007). Decision Neuroscience: New Directions in Studies of Judgment and Decision Making. Current Directions in Psychological Science, 16(3), 151-155.

The psychopath, the prisoner's dilemma and the invisible hand of morality

In the prisoner’s dilemma the police holds, in separate cells, two individuals accused of robbing a bank. The suspects (let’s call them Bob and Alice) are unable to communicate with each others. The police offers them the following options: confess or remain silent. If one confesses – implicating his or her partner – and the other remains silent, the former goes free while the other gets a 10 years sentence. If they both confess, they will serve a 5-years sentence. If they both remain silent, the sentence will be reduced to 2 years. The situation can be represented as the following payoff matrix:



Assuming that Bob and Alice have common knowledge – everybody knows that everybody knows that everybody knows, etc., ad infinitum – of each other’s rationality and the rules of the game, they should confess. Indeed, they will expect each other to make the best move, which is confessing, since confessing gives you either freedom or a 5-years sentence, while remaining silent entails either a 2-years or a 10-years sentence. The best reply to this move is also confessing, thus we expect Bob and Alice to confess. Even if they would be better-off in remaining silent, this choice is suboptimal: they risk a 10-years sentence if the other does not remain silent. In other words, they should not choose the cooperative move.

Experimental game theory indicates that subjects cooperate massively in prisoner’s dilemma. Recently, neuroeconomics showed that players enjoy cooperating, what economists refer to as the “warm glow of giving”. In the prisoner’s dilemma, players who initiate and players who experience mutual cooperation display activation in nucleus accumbens and other reward-related areas (Rilling et al. 2002).

In a new paper, Rilling and its collaborators (2007) now investigate how psychopathy influences cooperation in the prisoner's dilemma. Their subjects were not psychopaths per se: instead, they used normal individuals and--with a questionnaire--rated their attitudes on a "psychopathy scale". While in a scanner, they were then asked to play a prisoner's dilemma with nonscanned subjects. They were in fact playing against a computer following the "forgiving tit-for-tat" strategy", analogous to tit-for-tat excepts that it reciprocates previous defection only 67% of the time.

Behavioral results indicate that psychopathy is correlated with defection, even after mutual cooperation. One explanation could be that psychopaths have impaired amygdala, and hence are less sensible to aversive conditioning. This is coherent with fMRI data that suggests that the Cooperate-Defect outcome (I cooperate, you defect) elicit less aversive reaction in individual who score higher in psychopathy. Moreover, choosing to defect elicited more activity in the ACC and DLPFC (areas classically involved in emotional modulation and cognitive control), suggesting that defecting is effortful. Psychopathy, however, is correlated with less activity in these areas: it thus seems easier for psychopathic personalities to be non-cooperative. "Regular" people need more cognitive effort to override their cooperative biases.

fMRI suggest also that low-psychopathy and high-psychopathy subjects differs on how their brain implements cooperative behavior: while the formers rely on emotional biases (strong activation in the OFC, weak activation in DLPFC), the latters rely on cognitive control (weak activation in the OFC, strong activation in DLPFC). High-psychopathy subjects would be, according to Rilling et al., weakly emotionally biased toward defection: they exhibit a stronger OFC activation and a weaker DLPFC for defection. Thus, it seems that normal subjects tend to experiments the immediate gratification of cooperation, independently of the monetary payoff. Psychopaths do not feel the "warm glow" of cooperation, and thus do not cooperate.

Philosophically, there is an interesting lessons here: both low-psychopathy and high-psychopathy subjects follow their own, selfish biases: low-psychopathy ones enjoy cooperating, and high-psychopathy prefer defecting. This is consistent with a thesis I will one day describe more thoroughly another day, the "Invisible Hand of Morality": like markets, morality emerges out of the interaction of selfish agents. Luckily, thanks to evolution, culture, education, norms, etc., normal people selfishness tends to be geared toward cooperation. Psychopaths are not more selfish than normal people: their selfishness do not value cooperation, or other social virtues. Thus morality is not (only) "in the head": it is partly distributed is sensorimotor/somatovisceral mechanisms, cultural habits, external cues, institutions, etc. The other lesson is that morality is multi-realizable: it can be realized through emotional biases or cognitive control.

• Rilling, J. K., Glenn, A. L., Jairam, M. R., Pagnoni, G., Goldsmith, D. R., Elfenbein, H. A., et al. (2007). Neural correlates of social cooperation and non-cooperation as a function of psychopathy. Biological Psychiatry, 61(11), 1260-1271.
• Rilling, J., Gutman, D., Zeh, T., Pagnoni, G., Berns, G., & Kilts, C. (2002). A neural basis for social cooperation. Neuron, 35(2), 395-405.