Natural Rationality | decision-making in the economy of nature
Showing posts with label OFC. Show all posts
Showing posts with label OFC. Show all posts

9/13/07

Orbitofrontal Cortex Encodes Willingness to Pay in Everyday Economic Transactions

Last week, I discussed the confusion around the notion of valuation. Just to add a little complexity, a new study shows that OFC (also thought to encode economic value) encore the willingness-to-pay:
An essential component of every economic transaction is a willingness-to-pay (WTP) computation in which buyers calculate the maximum amount of financial resources that they are willing to give up in exchange for the object being sold. Despite its pervasiveness, little is known about how the brain makes this computation. We investigated the neural basis of the WTP computation by scanning hungry subjects' brains using functional magnetic resonance imaging while they placed real bids for the right to eat different foods. We found that activity in the medial orbitofrontal cortex and in the dorsolateral prefrontal cortex encodes subjects' WTP for the items. Our results support the hypothesis that the medial orbitofrontal cortex encodes the value of goals in decision making.




9/7/07

Why we need a neuroeconomic account of valuation

In my last post, I outlined an account of valuation. Whether mine is a good one is disputable, but the fact is that research in neuroeconomic, philosophy of mind, psychology, or any other field concerned with decision-making will sooner or later require a credible account of valuation, and value. Here is two reasons why.

First, there is a significant overlap between brain areas and processes in different valuations domains. While neuroeconomics, neuroethics and neuropolitics began to make explicit the neural mechanisms involved in these domains (Glimcher, 2003; Tancredi, 2005; Westen, 2007), attempts to cross-fertilize research are scarce. Research showed that economic, moral and political cognition involve similar brain processes. For instance, whether subjects play economic games (Rilling et al., 2002; Sanfey et al., 2003), reflect upon moral issues (Greene & Haidt, 2002; Koenigs et al., 2007) or make political judgments (Kaplan et al., 2007; Knutson et al., 2006; Westen et al., 2006), these tasks recruits principally the following evaluative mechanisms: Core affect, monitoring and control mechanisms described in my last post.

Second, even in one area--neuroeconomics--it is not clear what researchers mean when they talk about the "neural substrate of economic value". Take for instance two recent studies. Seo et al (2007) and Padoa-Schioppa (2007) both attempt to identify the brain valuation processes. The first one conclude that

A rich literature from lesion studies, functional imaging, and primate neurophysiology suggests that critical mechanisms for economic choice might take place in the orbitofrontal cortex. More specifically, recent results from single cell recordings in monkeys link OFC [Orbitofrontal Cortex] to the computation of economic value. We showed that the value representation in OFC reflects the subjective nature of economic value, and that neurons in this area encode value per se, independently of the visuo-motor contingencies of choice

The other discuss how the DLPFC contribute to decision-making:

individual neurons in the dorsolateral prefrontal cortex (DLPFC) encoded 3 different types of signals that can potentially influence the animal's future choices. First, activity modulated by the animal's previous choices might provide the eligibility trace that can be used to attribute a particular outcome to its causative action. Second, activity related to the animal's rewards in the previous trials might be used to compute an average reward rate. Finally, activity of some neurons was modulated by the computer's choices in the previous trials and may reflect the process of updating the value functions.
So how is something valuated? DLPFC or OFC ? How exactly they differ? Yes, one is about "reward" and the other "economic value", but again, since money can be rewarding and food can have an economic value (utility), it is not clear that different words refer to different processes. On top of that, there is also a huge literature on dopaminergic systems and valuation (see Montague et al, 2006; Montague, 2006) for a complete review). So some clarification is required here. I will try, in future post, do discuss these questions.


  • Glimcher, P. W. (2003). Decisions, uncertainty, and the brain : the science of neuroeconomics. Cambridge, Mass. ; London: MIT Press.
  • Greene, J. D., & Haidt, J. (2002). How (and where) does moral judgment work? Trends Cogn Sci, 6(12), 517-523.
  • Kaplan, J. T., Freedman, J., & Iacoboni, M. (2007). Us versus them: Political attitudes and party affiliation influence neural response to faces of presidential candidates. Neuropsychologia, 45(1), 55-64.
  • Knutson, K. M., Wood, J. N., Spampinato, M. V., & Grafman, J. (2006). Politics on the Brain: An fMRI Investigation. Soc Neurosci, 1(1), 25-40.
  • Koenigs, M., Young, L., Adolphs, R., Tranel, D., Cushman, F., Hauser, M., et al. (2007). Damage to the prefrontal cortex increases utilitarian moral judgements. Nature, 446(7138), 908-911.
  • Montague, P. R., King-Casas, B., & Cohen, J. D. (2006). Imaging valuation models in human choice. Annu Rev Neurosci, 29, 417-448.
  • Montague, R. (2006). Why choose this book? : how we make decisions. New York: Penguin Group.
    Padoa-Schioppa, C. (2007). Orbitofrontal Cortex and the Computation of Economic Value. Ann NY Acad Sci, annals.1401.1011.
  • Rilling, J., Gutman, D., Zeh, T., Pagnoni, G., Berns, G., & Kilts, C. (2002). A neural basis for social cooperation. Neuron, 35(2), 395-405.
  • Sanfey, A. G., Rilling, J. K., Aronson, J. A., Nystrom, L. E., & Cohen, J. D. (2003). The neural basis of economic decision-making in the Ultimatum Game. Science, 300(5626), 1755-1758.
  • Seo, H., Barraclough, D. J., & Lee, D. (2007). Dynamic Signals Related to Choices and Outcomes in the Dorsolateral Prefrontal Cortex. Cereb. Cortex, 17(suppl_1), i110-117
  • Tancredi, L. R. (2005). Hardwired behavior : what neuroscience reveals about morality. New York: Cambridge University Press.
  • Westen, D. (2007). The political brain : the role of emotion in deciding the fate of the nation. New York: PublicAffairs.
  • Westen, D., Blagov, P. S., Harenski, K., Kilts, C., & Hamann, S. (2006). Neural Bases of Motivated Reasoning: An fMRI Study of Emotional Constraints on Partisan Political Judgment in the 2004 U.S. Presidential Election. J. Cogn. Neurosci., 18(11), 1947-1958.



A neuroecomic picture of valuation



Values are everywhere: ethics, politics, economics, law, for instance, all deal with values. They all involve socially situated decision-makers compelled to make evaluative judgment and act upon them. These spheres of human activity constitute three aspects of the same problem, that is, guessing how 'good' is something. This means that value 'runs' on valuation mechanisms. I suggest here a framework to understand valuation.

I define here valuation as the process by which a system maps an object, property or event X to a value space, and a valuation mechanism as the device implementing the matching between X and the value space. I do not suggest that values need to be explicitly represented as a space: by valuation space, I mean an artifact that accounts for the similarity between values by plotting each of them as a point in a multidimensional coordinate system. Color spaces, for instance, are not conscious representation of colors, but spatial depiction of color similarity along several dimensions such as hue, saturation brightness.

The simplest, and most common across the phylogenetic spectrum, value space has two dimensions: valence (positive or negative) and magnitude. Valence distinguishes between things that are liked and things that are not. Thus if X has a negative valence, it does not implies that X will be avoided, but only that it is disliked. Magnitude encodes the level of liking vs. disliking. Other dimensions might be added—temporality (whether X is located in the present, past of future), other- vs. self-regarding, excitatory vs. inhibitory, basic vs. complex, for instance—but the core of any value system is valence and magnitude, because these two parameters are required to establish rankings. To prefer Heaven to Hell, Democrats to Republicans, salad to meat, or sweet to bitter involves valence and magnitude.

Nature endowed many animals (mostly vertebrates) with fast and intuitive valuation mechanisms: emotions.[1] Although it is a truism in psychology and philosophy of mind and that there is no crisp definition of what emotions are[2], I will consider here that an emotion is any kind of neural process whose function is to attribute a valence and a magnitude to something else and whose operative mode are somatic markers. Somatic markers[3] are bodily states that ‘mark’ options as advantageous/disadvantageous, such as skin-conductance, cardiac rhythm, etc. Through learning, bodily states become linked to neural representations of the stimuli that brought these states. These neural structures may later reactivate the bodily states or a simulation of these states and thereby indicate the valence and magnitude of stimuli. These states may or may not account for many legitimate uses of the word “emotions”, but they constitute meaningful categories that could identify natural kinds[4]. In order to avoid confusion between folk-psychological and scientific categories, I will rather talk of affects and affective states, not emotions.

More than irrational passions, affectives states are phylogenetically ancient valuation mechanisms. Since Darwin,[5] many biologists, philosophers and psychologists[6] have argued that they have adaptive functions such as focusing attention and facilitating communication. As Antonio Damasio and his colleagues discovered, subjects impaired in affective processing are unable to cope with everyday tasks, such as planning meetings[7]. They lose money, family and social status. However, they were completely functional in reasoning or problem-solving tasks. Moreover, they did not felt sad for their situation, even if they perfectly understood what “sad” means, and seemed unable to learn from bad experiences. They were unable to use affect to aid in decision-making, a hypothesis that entails that in normal subjects, affect do aid in decision-making. These findings suggest that decision-making needs affect, not as a set of convenient heuristics, but as central evaluative mechanisms. Without affects, it is possible to think efficiently, but not to decide efficiently (affective areas, however, are solicited in subjects who learn to recognize logical errors[8]).

Affects, and specially the so-called ‘basic’ or ‘core’ ones such as anger, disgust, liking and fear[9] are prominent explanatory concepts in neuroeconomics. The study of valuation mechanisms reveals how the brain values certain objects (e.g. money), situations (e.g. investment, bargaining) or parameter (risk, ambiguity) of an economic nature. Three kinds of mechanisms are typically involved in neuroeconomic explanations:

  1. Core affect mechanisms, such as fear (amygdala), disgust (anterior insula) and pleasure (nucleus accumbens), encode the magnitude and valence of stimuli.
  2. Monitoring and integration mechanisms (ventromedial/mesial prefrontal, orbitofrontal cortex, anterior cingulate cortex) combine different values and memories of values together
  3. Modulation and control mechanisms (prefrontal areas, especially the dorsolateral prefrontal cortex), modulate or even override other affect mechanisms.

Of course, there is no simple mapping between psychological functions and neural structures, but cognitive and affective neuroscience assume a dominance and a certain regularity in functions. Disgust does not reduce to insular activation, but anterior insula is significantly involved in the physiological, cognitive and behavioral expressions of disgust. There is a bit of simplification here—due to the actual state of science—but enough to do justice to our best theories of brain functioning. I will here review two cases of individual and strategic decision-making, and will show how affective mechanisms are involved in valuation[10].

In a study by Knutson et al.,[11] subjects had to choose whether or not they would purchase a product (visually presented), and then whether or not they would buy it at a certain price. While desirable products caused activation in the nucleus accumbens, activity is detected in the insula when the price is seen as exaggerated. If the price is perceived as acceptable, a lower insular activation is detected, but mesial prefrontal structures are more solicited. The activation in these areas was a reliable predictor of whether or not subjects would buy the product: prefrontal activation predicted purchasing, while insular activation predicted the decision of not purchasing. Thus purchasing decision involves a tradeoff, mediated by prefrontal areas, between the pleasure of acquiring (elicited in the nucleus accumbens) and the pain of purchasing (elicited in the insula). A chocolate box—a stimulus presented to the subjects—is located in the high-magnitude, positive-valence regions of the value space, while the same chocolate box priced at $80 is located in the high-magnitude, negative-valence regions of the space.

In the ultimatum game, a ‘proposer’ makes an offer to a ‘responder’ that can either accept or refuse the offer. The offer is a split of an amount of money. If the responder accepts, she keeps the offered amount while the proposer keeps the difference. If she rejects it, however, both players get nothing. Orthodox game theory recommends that proposers offer the smallest possible amount, while responder should accept every proposition, but all studies confirm that subjects make fair offer (about 40% of the amount) and reject unfair ones (less than 20%)[12]. Brain scans of people playing the ultimatum game indicate that unfair offers trigger, in the responders’ brain, a ‘moral disgust’: the anterior insula is more active when unfair offers are proposed,[13] and insular activation is proportional to the degree of unfairness and correlated with the decision to reject unfair offers[14]. Moreover, unfair offers are associated with greater skin conductance[15]. Visceral and insular responses occur only when the proposer is a human: a computer does not elicit those reaction. Beside the anterior insula, two other areas are recruited in ultimatum decisions: the dorsolateral prefrontal cortex (DLPFC). When there is more activity in the anterior insula than in the DLPFC, unfair offers tend to be rejected, while they tend to be accepted when DLPFC activation is greater than anterior insula.

These two experiments illustrate how neuroeconomics begin to decipher the value spaces and how valuation relies on affective mechanisms. Although human valuation is more complex than the simple valence-magnitude space, this ‘neuro-utilitarist’ framework is useful for interpreting imaging and behavioral data: for instance, we need an explanation for insular activation in purchasing and ultimatum decision, and the most simple and informative, as of today, is that it trigger a simulated disgust. More generally, it also reveals that the human value space is profoundly social: humans value fairness and reciprocity. Cooperation[16] and altruistic punishment[17] (punishing cheaters at a personal cost when the probability of future interactions is null), for instance, activate the nucleus accumbens and other pleasure-related areas. People like to cooperate and make fair offers.

Neuroeconomic experiments also indicate how value spaces can be similar across species. It is known for instance that in humans, losses[18] elicit activity in fear-related areas such as amygdala. Since capuchin monkey’s behavior also exhibit loss-aversion[19] (i.e., a greater sensitivity to losses than to equivalent gains), behavioral evidence and neural data suggests that the neural implementation of loss-aversion in primates shares common valuation mechanisms and processing. The primate—and maybe the mammal or even the vertebrate—value space locate loss in a particular region.

Notes

  • [1] (Bechara & Damasio, 2005; Bechara et al., 1997; Damasio, 1994, 2003; LeDoux, 1996; Naqvi et al., 2006; Panksepp, 1998)
  • [2] (Faucher & Tappolet, 2002; Griffiths, 2004; Russell, 2003)
  • [3] (Bechara & Damasio, 2005; Damasio, 1994; Damasio et al., 1996)
  • [4] (Griffiths, 1997)
  • [5] (Darwin, 1896)
  • [6] (Cosmides & Tooby, 2000; Paul Ekman, 1972; Griffiths, 1997)
  • [7] (Damasio, 1994)
  • [8] (Houde & Tzourio-Mazoyer, 2003)
  • [9] (Berridge, 2003; P. Ekman, 1999; Griffiths, 1997; Russell, 2003; Zajonc, 1980)
  • [10] The material for this part is partly drawn from (Hardy-Vallée, forthcoming)
  • [11] (Knutson et al., 2007).
  • [12] (Oosterbeek et al., 2004)
  • [13] (Sanfey et al., 2003)
  • [14] (Sanfey et al., 2003: 1756)
  • [15] (van 't Wout et al., 2006)
  • [16] (Rilling et al., 2002).
  • [17] (de Quervain et al., 2004).
  • [18] (Naqvi et al., 2006)
  • [19] (Chen et al., 2006)

References

  • Bechara, A., & Damasio, A. R. (2005). The somatic marker hypothesis: A neural theory of economic decision. Games and Economic Behavior, 52(2), 336.
  • Bechara, A., Damasio, H., Tranel, D., & Damasio, A. R. (1997). Deciding Advantageously Before Knowing the Advantageous Strategy. Science, 275(5304), 1293-1295.
  • Berridge, K. C. (2003). Pleasures of the brain. Brain and Cognition, 52(1), 106.
  • Chen, M. K., Lakshminarayanan, V., & Santos, L. (2006). How Basic Are Behavioral Biases? Evidence from Capuchin Monkey Trading Behavior. Journal of Political Economy, 114(3), 517-537.
  • Cosmides, L., & Tooby, J. (2000). Evolutionary psychology and the emotions. Handbook of Emotions, 2, 91-115.
  • Damasio, A. R. (1994). Descartes' error : emotion, reason, and the human brain. New York: Putnam.
  • Damasio, A. R. (2003). Looking for Spinoza : joy, sorrow, and the feeling brain (1st ed.). Orlando, Fla. ; London: Harcourt.
  • Damasio, A. R., Damasio, H., & Christen, Y. (1996). Neurobiology of decision-making. Berlin ; New York: Springer.
  • Darwin, C. (1896). The expression of the emotions in man and animals ([Authorized ed.). New York,: D. Appleton.
  • de Quervain, D. J., Fischbacher, U., Treyer, V., Schellhammer, M., Schnyder, U., Buck, A., et al. (2004). The neural basis of altruistic punishment. Science, 305(5688), 1254-1258.
  • Ekman, P. (1972). Emotion in the human face: guide-lines for research and an integration of findings. New York,: Pergamon Press.
  • Ekman, P. (1999). Basic emotions. In T. Dalgleish & M. Power (Eds.), Handbook of Cognition and Emotion (pp. 45-60). Sussex John Wiley & Sons, Ltd.
  • Faucher, L., & Tappolet, C. (2002). Fear and the focus of attention. Consciousness & emotion, 3(2), 105-144.
  • Griffiths, P. E. (1997). What emotions really are : the problem of psychological categories. Chicago, Ill.: University of Chicago Press.
  • Griffiths, P. E. (2004). Emotions as Natural and Normative Kinds. Philosophy of Science, 71, 901–911.
  • Hardy-Vallée, B. (forthcoming). Decision-making: a neuroeconomic perspective. Philosophy Compass.
  • Houde, O., & Tzourio-Mazoyer, N. (2003). Neural foundations of logical and mathematical cognition. Nature Reviews Neuroscience, 4(6), 507-514.
  • Knutson, B., Rick, S., Wimmer, G. E., Prelec, D., & Loewenstein, G. (2007). Neural predictors of purchases. Neuron, 53(1), 147-156.
  • LeDoux, J. E. (1996). The emotional brain : the mysterious underpinnings of emotional life. New York: Simon & Schuster.
  • Naqvi, N., Shiv, B., & Bechara, A. (2006). The Role of Emotion in Decision Making: A Cognitive Neuroscience Perspective. Current Directions in Psychological Science, 15(5), 260-264.
  • Oosterbeek, H., S., R., & van de Kuilen, G. (2004). Differences in Ultimatum Game Experiments: Evidence from a Meta-Analysis. Experimental Economics 7, 171-188.
  • Panksepp, J. (1998). Affective neuroscience : the foundations of human and animal emotions. New York: Oxford University Press.
  • Rilling, J. K., Gutman, D. A., Zeh, T. R., Pagnoni, G., Berns, G. S., & Kilts, C. D. (2002). A Neural Basis for Social Cooperation. Neuron, 35(2), 395-405.
  • Russell, J. A. (2003). Core affect and the psychological construction of emotion. Psychol Rev, 110(1), 145-172.
  • Sanfey, A. G., Rilling, J. K., Aronson, J. A., Nystrom, L. E., & Cohen, J. D. (2003). The neural basis of economic decision-making in the Ultimatum Game. Science, 300(5626), 1755-1758.
  • van 't Wout, M., Kahn, R. S., Sanfey, A. G., & Aleman, A. (2006). Affective state and decision-making in the Ultimatum Game. Exp Brain Res, 169(4), 564-568.
  • Zajonc, R. B. (1980). Feeling and thinking: Preferences need no inferences. American Psychologist, 35(2), 151-175.





7/19/07

Beautiful picture of brain areas involved in decision-making

Found yesterday, in a paper by Sanfey (nice review paper, by the way):




"Fig. 2. Map of brain areas commonly found to be activated in decision-making studies. The sagittal section (A) shows the location of the anterior cingulate cortex (ACC), medial prefrontal cortex (MPFC), orbitofrontal cortex (OFC), nucleus accumbens (NA), and substantia nigra (SN). The lateral view (B) shows the location of the dorsolateral prefrontal cortex (DLPFC) and lateral intraparietal area (LIP). The axial section (C; cut along the white line in A and B) shows the location of the insula (INS) and basal ganglia (BG)."
from:



6/30/07

Must-read reviews

The 2007 edition of the Annual Review of Neuroscience (vol. 30, 2007) is online since today. 


Although I did not read them yet, anybody interested in decision-making, neuroeconomics or anything this blog discusses should read this two papers:

(I like explicit titles such as these! )

The Annual Reviews is a series of  "authoritative, analytic reviews in 33 focused disciplines within the Biomedical, Physical, and Social Science" (from the publisher's website). The state-of-the art in any of these. Enjoy !



3/28/07

Anterior cingulate choices and orbitofrontal preferences

With all the studies in neuroeconomics, it is hard to get the whole picture of decision-making. In a paper in Trends in Cognitive Science, Rushworth et al. review the contribution of two important areas: the anterior cingulate cortex (ACC) and orbitofrontal cortex (OFC). After reviewing many studies, the authors conclude that they substantially contribute , respectively, to the generation of reward-based action and to representation of value. The OFC thus encodes values, expectations and preferences (patient with OFC lesions are impaired in their decision-making abilities because they cannot asses the utility of different options). The ACC is more concerned about the values of action and the generation of exploratory actions and their valuation. Thus, to make an extremely simplistic description, OFC is about preferences and ACC about choices, the two most important components of decision making: in making a rational decision, one chooses to do A because one prefers A to other options:

The OFC is important when reinforcement is associated with stimuli and for the representation of preferences. It is critical when behaviour depends on detailed, flexible and adjustable predictions of outcomes or on models of the reinforcement environment. In the ACC, reward representation is closely bound to action or task representation. This means that the ACC mediates the relationship between the previous action-reinforcement history and the next action choice.
ACC is also more involved in social cognition.

The following image depicts the connections between OFC, ACC and other areas. As you can see, OFC is a little more on the "input side" while ACC is on the "ouptut side". In both cases, the amygdala (involved in fear, memory, learning and attention) and the ventral striatum (reward processing and motivation) are important players in this game:

We are far from having the whole picture, or a neuroeconomic Theory of Everything, but these syntheses help understanding the mechanisms of decision-making. The next big step, I guess, would be the integration of this connectivity pattern with the function of dopaminergic neurons, thought to implements TD-learning algorithms (see this previous post)

In any case, whatever will be the details, it is clear that a theory of decision-making will be a theory of "affective management". In a historical-philosophical perspective, all these researches can be seen as a reactualization of the intellectualism/voluntarism dispute According to intellectualism, a rational action is the product of a reasoning process that determines what is good, while voluntarism take the action as the product of a motivation. Neuroeconomics, hedonic psychology and affective cognition all suggest a contemporary form of voluntarism.