Anterior cingulate choices and orbitofrontal preferences

With all the studies in neuroeconomics, it is hard to get the whole picture of decision-making. In a paper in Trends in Cognitive Science, Rushworth et al. review the contribution of two important areas: the anterior cingulate cortex (ACC) and orbitofrontal cortex (OFC). After reviewing many studies, the authors conclude that they substantially contribute , respectively, to the generation of reward-based action and to representation of value. The OFC thus encodes values, expectations and preferences (patient with OFC lesions are impaired in their decision-making abilities because they cannot asses the utility of different options). The ACC is more concerned about the values of action and the generation of exploratory actions and their valuation. Thus, to make an extremely simplistic description, OFC is about preferences and ACC about choices, the two most important components of decision making: in making a rational decision, one chooses to do A because one prefers A to other options:

The OFC is important when reinforcement is associated with stimuli and for the representation of preferences. It is critical when behaviour depends on detailed, flexible and adjustable predictions of outcomes or on models of the reinforcement environment. In the ACC, reward representation is closely bound to action or task representation. This means that the ACC mediates the relationship between the previous action-reinforcement history and the next action choice.

ACC is also more involved in social cognition.

The following image depicts the connections between OFC, ACC and other areas. As you can see, OFC is a little more on the "input side" while ACC is on the "ouptut side". In both cases, the amygdala (involved in fear, memory, learning and attention) and the ventral striatum (reward processing and motivation) are important players in this game:

We are far from having the whole picture, or a neuroeconomic Theory of Everything, but these syntheses help understanding the mechanisms of decision-making. The next big step, I guess, would be the integration of this connectivity pattern with the function of dopaminergic neurons, thought to implements TD-learning algorithms (see this previous post)

In any case, whatever will be the details, it is clear that a theory of decision-making will be a theory of "affective management". In a historical-philosophical perspective, all these researches can be seen as a reactualization of the intellectualism/voluntarism dispute According to intellectualism, a rational action is the product of a reasoning process that determines what is good, while voluntarism take the action as the product of a motivation. Neuroeconomics, hedonic psychology and affective cognition all suggest a contemporary form of voluntarism.

A Neuropolitic look at political psychology

In Politics on the Brain: An fMRI Investigation, (a paper forthcoming in Social Neuroscience) Knutson et al. shows that political preferences (whether you prefer George Bush or Hilary Clinton) recruits 2 different neural circuits: one rapid, stereotypic, and emotional - ventromedial PFC and amygdala- and the other more deliberative - anterior prefrontal activations. When subjects were shown a image of a politician faces, both systems fire. Thus it seems that liking/disliking a politician is an affective reaction modulated by other knowledge, probably political values. The strength of affiliation with a party (in this case, Democrat vs. Republican) correlated negatively with PFC activation: thus emotional markers are not principally signs of political orientation, but signs of personal affiliation. Political orientation may modulate personal orientation when, for instance, one agree with a party without endorsing its leader's opinions. Somewhere in Man's Fate, Andre Malraux wrote (I quote approximatively, I cannot find the original quote): "men that follows ideas, in the end, always follows individuals". Most of our cognition is affective, and most of our affections are social. This is why you will always see politician shaking hands, kissing babies and smiling: they recruit brain structures involved in social-affective cognition. Political education, then, should consist in learning how to override these emotional reaction, and thus research on emotional learning are important for democracy.
Another related study might be of interest for neuropolitics int he last edition of Social Cognitive and Affective Neuroscience. An article on the neural mechanisms of social fear transmission concludes that learned fear can be "as powerful as fears originating from direct experiences." Thus emotional education (a form of emotional learning) can induce genuine emotions - at least fear.
Together, these studies suggest that "affective political cognition" could be not only theoretically interesting, but highly important for policy making.

The philosophy of preferences

Preferences and Well-Being is the subject of the new edition of the Royal Institute of Philosophy Supplements . Philosophers discuss the different theoretical issues related to preference, rationality, decision, etc. Worth reading, although a serious lack of experimental data flaws many of these conceptual endeavors.

A dual account of addiction?

In a recent edition of The American Journal of Bioethics, Steven Hyman presents the recent findings on the neurobiology and addiction (a clear and complete review, by the way), especially cognitive neuroscience, and how these researches could shed light on addiction, its control and its cure. Against the 'disease model' of addiction, that consider addiction as completely out of control, and the folk psychological model of voluntary action, according to which we have a conscious control over most of our actions, he shows that a more complex picture should be put forth. Yes, addiction is partly out of control, as reward-seeking mechanisms are hijacked by drugs of abuse and cognitive control mechanisms are impaired, but in appropriate environmental settings, relatives or friends may act as 'external prefrontal cortex' and help the addicted indiviual. Hyman's model of addiction seems to endorse (although he does not mention it explicitly) a dual-mechanisms account of cognition (the idea that the cognition operates through 2 systems, one fast/intuitive/automatic/parrallel, the other slow/reflected/controlled/serial). His conclusion aptly summarizes his point:

/>

Mechanisms that evolved to motivate survival behaviors, the pursuit of natural rewards, are usurped by the potent and abnormal dopamine signal produced by addictive drugs. The result is a brain in which drug cues powerfully activate drug seeking, and in which attempts to suppress drugseeking result in intense craving. This model does not, however, reduce addicted individuals to zombies who are permanently controlled by external cues. As overvalued as drugs become, as potent as the effects of drug cues on behavior, other goals are not extirpated. Perhaps in a drug-free context, perhaps with a good measure of initial coercion, perhaps with family, friends, and caregivers acting as external "prostheses" to strengthen and partially replace damaged frontal mechanisms of cognitive control, and often despite multiple relapses, addicts can cease drug use and regain a good measure of control over their drug taking. Our current models help explain why recovery is difficult and why relapses occur even long after detoxification and rehabilitation. The long experience of humanity with addiction does not counsel fatalism, but implacable efforts to overcome the behavioral effects of neural circuits hijacked by drugs. Finally, views based on cognitive neuroscience and studies of addiction pathogenesis suggest that some apparently voluntary behaviors may not be as freely planned and executed as they first appear. Such cognitive views have not yet penetrated folk psychology, and it is premature for these views to have any place in the courtroom (Morse 2004a; Greene and Cohen 2004). Nonetheless these cognitive views deserve a place in current ethical discussions of personal responsibility. For many reasons, it may be wise for societies to err on the side of holding addicted individuals responsible for their behavior and to act as if they are capable of exerting more control than perhaps they can; however, if the ideas expressed in this review are right, it should be with a view to rehabilitation of the addicted person and protection of society rather than moral opprobrium.

Technorati Tags: addiction, decision, control, brain

Neuroscience and moral decision-making - 2 new studies

First, a great review in Nature Reviews Neuroscience on the neural basis of punishment. Their conclusion is that the so-called 'Strong Reciprocity' explanation of altruistic punishment, the idea put forth by behavioral economist to explain why subjects tend to punish 'cheaters' in experimental games, event at their own cost, may not be that hardwired.

there is no reason to assume that altruistic punishment should necessarily be hard-wired as an inherited intrinsic motivational goal (that is, as an unconditioned appetitive stimulus) in the same manner as primary rewards. However, neither does it exclude the possibility. Future research might help to resolve both the role of learning and early development in the acquisition of altruistic behaviour.

Second, a new Nature paper from Hauser, Adolphs, Damasio and other collaborators on the consequences of ventromedial prefrontal cortex (VMPC) lesions in moral reasoning. This area is particularly important in the processing of emotional information and in anticipating emotional event. People with damages VMPFC tend to lose risk-aversion, because they lose the ability to anticipate negative feeling, and thus they engage in behaviors detrimental to their own well-being (loss of money, friends, family, social status, see Damasio's Descartes' Error), even if their intellectual faculty are perfectly normal. They also lose, according to this new study, the ability to use these emotional predictions for moral judgment. Normal subjects usually have a deontological conception of morality (i.e., there are moral principles that applies come what may) in certain cases, such as as strangling a crying baby in order to prevent it from revealing your position while an enemy troops wander in your village, with the instruction to kill all civilians. Normal subjects have a strong emotional response that leads them to refuse. VMPFC-imparied subjects, however tend to rely a little more on a 'utilitarian' scheme: since they lack the emotional appraisal of the situation, they use a cost-benefit reasoning. Thus, once again, neuroscience shows how emotions are important for moral and reasoning. The important question, of course, is how these studies can help us to understand how we should behave. Interesting question for ethical neurophilosophy...

A short primer on dopamine and TD learning

Many researches indicates that dopaminergic systems have an important role in decision-making, and that their activity can be precisely formulated by TD algorithms. Here is a brief description, from a forthcoming paper of mine:
--

According to many findings, utility computation is realized by dopaminergic systems, a network of structures in ‘older’ brain areas highly involved in motivation and valuation (Berridge, 2003; Montague & Berns, 2002). Neuroscience revealed their role in working memory, motivation, learning, decision-making planning and motor control (Morris et al., 2006). Dopaminergic neurons are activated by stimuli related to primary rewards (juice, food) and stimuli that recruit attention (new or intense). It is important to note that they do not encode hedonic experiences, but predictions of expected reward. Activations of dopaminergic neurons respond selectively to prediction errors: the presence of unexpected reward or the absence of expected reward. In other words, they detect the discrepancies between predicted and experienced utility. Moreover, dopaminergic neurons learns from their mistake: from these prediction errors they learn to predict future rewarding events and can then bias action choice. Computational neuroscience identified a class of reinforcement learning algorithms that mirror the activity of dopaminergic activity (Niv et al., 2005; Suri & Schultz, 2001). It is suggested that dopaminergic neurons broadcast in different brain areas a reward-predictio error signal similar to those displayed by temporal difference (TD) algorithms developed by computer scientists (Sutton & Barto, 1987, 1998). This dopaminergic mechanisms use sensory inputs to predict future rewards. The difference between successive value predictions is computed and constitutes an error signal. The model then updates a value function (the function that maps state-action pairs to numerical values) according to the prediction error. Thus TD-learning algorithms are neural mechanisms of decision-making under uncertainty implemented in dopaminergic systems. They are not involved only in basic reward prediction, such as food, but also abstract stimuli like art, branded good, love or trust (Montague et al., 2006, p. 420). From the mouthwatering vision of a filet mignon in a red wine sauce to the intellectual contemplation of Any Warhol’s Pop Art Brillo Boxes, the valuation mechanisms are essentially the same

Berridge, K. C. (2003). Irrational pursuits: Hyper-incentives from a visceral brain. In I. Brocas & J. Carrillo (Eds.), The psychology of economic decisions (pp. 17-40). Oxford: Oxford University Press.
Montague, P. R., & Berns, G. S. (2002). Neural economics and the biological substrates of valuation. Neuron, 36(2), 265-284.
Montague, P. R., King-Casas, B., & Cohen, J. D. (2006). Imaging valuation models in human choice. Annu Rev Neurosci, 29, 417-448.
Morris, G., Nevet, A., Arkadir, D., Vaadia, E., & Bergman, H. (2006). Midbrain dopamine neurons encode decisions for future action. Nat Neurosci, 9(8), 1057-1063.
Niv, Y., Duff, M. O., & Dayan, P. (2005). Dopamine, uncertainty and td learning. Behav Brain Funct, 1, 6.
Suri, R. E., & Schultz, W. (2001). Temporal difference model reproduces anticipatory neural activity. Neural Comput, 13(4), 841-862.
Sutton, R. S., & Barto, A. G. (1987). A temporal-difference model of classical conditioning. Paper presented at the Ninth Annual Conference of the Cognitive Science Society.
Sutton, R. S., & Barto, A. G. (1998). Reinforcement learning : An introduction. Cambridge, Mass.: MIT Press.

Monkey Moral

An excellent paper in the New York Times about the evolutionary precursors of morality. Primatologist Frans de Waal research is presented and commentend by many philosophers.





Technorati Tags: moral, evolutions

Neuroeconomics - an overview

I put my notes online here on the (recent) history of neuroeconomics, its conceptual roots, and some important findings, plus a bibliography. This should give you a broad overview of the field.