Why Neuroeconomics Needs a Concept of (Natural) Rationality

Neuroeconomists (more than “decision neuroscientists”) often report their finding as strong evidence against the rationality of decision-makers. In the case of cooperation it is often claimed that emotions motivate cooperation since neural activity elicited by cooperation overlaps with neural activity elicited by hedonic rewards (Fehr & Camerer, 2007). Also, when subjects have to choose whether or not they would purchase a product, desirable products cause activation in the nucleus accumbens (associated with anticipation of pleasure). However, if the price is seen as exaggerated, activity is detected in the insula (involved in disgust and fear; Knutson 2007).

The accumulation of evidence about the engagement of affective areas in decison-making is undisputable, and seems to make a strong case against a once pervasive “rationalist” vision of decision-making in cognitive science and economics. This is not, however, a definitive argument for emotivism (we choose with our "gut feelings") and irrationalism. For at least three reasons (methodological, empirical and conceptual), these findings should not be seen as supporting an emotivist account.

First, characterizing a brain area as “affective” or “emotional” is misleading. There is no clear distinction, in the brain, between affective and cognitive areas. For instance, the anterior insula is involved in disgust, but also in disbelief (Harris et al., 2007). A high-level task such as cognitive control (e.g. holding items in working memory in a goal-oriented task) requires both “affective” and “cognitive” areas (Pessoa, 2008). The affective/cognitive distinction is a folk-psychological one, not a reflection of brain anatomy and connectivity. There is a certain degree of specialization, but generally speaking any task recruits a wide arrays of areas, and each area is redeployed in many tasks. In complex being like us, so-called “affective” areas are never purely affective: they always contribute to higher-level cognition, such as logical reasoning (Houde & Tzourio-Mazoyer, 2003). Similarly, while the amygdala has been often described as a “fear center”, its function is much more complex, as it modulates emotional information, react to unexpected stimuli and is heavily recruited in visual attention, a “cognitive” function. It is therefore wrong to consider “affective” areas as small emotional agents that are happy or sad and make us happy of sad. Instead of employing folk-psychological categories, their functional contribution should be understood in computational terms: how they process signals, how information is routed between areas and how they affect behavior and thought.

Second, even if there are affective areas, they are always complemented or supplemented by “cognitive” ones: the dorsolateral prefrontal cortex (DLPFC) for instance (involved in cognitive control and goal maintenance), is recruited in almost all decision-making task, and has been shown to be involved in norm-compliant behavior and purchasing decisions. In the ultimatum game, beside the anterior insula, two other areas are recruited: the DLPFC and the anterior cingulate cortex (ACC), involved in cognitive conflict and emotional modulation. Explainiations of ultimatum decisions spell out neural information-processing mechanisms, not “emotions”.

Check for instance the neural circuitry involved in cognitive control: you would think it is only prefrontal areas, but as it turns out, "cognitive" and "affective" area sare required for this competence:

[Legend: This extended control circuit contains traditional control areas, such as the anterior cingulate cortex (ACC) and the lateral prefrontal cortex (LPFC), in addition to other areas commonly linked to affect (amygdala) and motivation (nucleus accumbens). Diffuse, modulatory effects are shown in green and originate from dopamine-rich neurons from the ventral tegmental area (VTA). The circuit highlights the cognitive–affective nature of executive control, in contrast to more purely cognitive-control proposals. Several connections are not shown to simplify the diagram. Line thickness indicates approximate connection strength. OFC, orbitofrontal cortex.From Pessoa, 2008]

As Michael Anderson pointed out in a series of papers (2007a and b, among others), there is many-to-many mapping between brain functions and cognitive functions. So the concept of "emotional areas" should be banned from neuroeconomics vocabulary before it is too late.

Third, a point that has been neglected by many research about decision-making neural activation of a particular brain area is always explanatory with regard to its contribution in understanding personal-level properties. If we learn that the anterior insula react to unfair offers, we are not singling out the function of this area, but explaining how the person’s decision is responsive to a particular type of valuation. The basic unit of analysis of decisions is not neurons, but judgments. We may study sub-judgmental (e.g. neural) mechanisms and how they contribute to judgment formation; or we may study supra-judgmental mechanisms (e.g. reasoning) and how they articulate judgments. Emotions, as long as they are understood as affective reactions, are not judgments: they either contribute to judgments or are construed as judgments. In both case, the category “emotions” seems superfluous for explaining the nature of the judgment itself. Thus, if judgments are the basic unit of analysis, brain areas are explanatory insofar as they make explicit how individuals arrive at a certain judgment, how it is implemented, etc: what kind of neural computations are carried out? Take, for example, cooperation in the prisoner's dilemma. Imaging studies show that when high-psychopathy and low-psychopathy subjects choose to cooperate, different neural activity is observed: the former use more prefrontal areas than the latter, indicating that cooperation is more efforful (see this post). This is instructive: we learn something about the information- processing not about "emotions" or "reason".

In the end, we want to know how these mechanisms fix beliefs, desires and intentions: neuroeconomics can be informative as long as it aims at deciphering human natural rationality.


References

• Anderson, M. L. (2007a). Evolution of Cognitive Function Via Redeployment of Brain Areas. Neuroscientist, 13(1), 13-21.
• Anderson, M. L. (2007b). The Massive Redeployment Hypothesis and the Functional Topography of the Brain. Philosophical Psychology, 20(2), 143 - 174.
• Fehr, E., & Camerer, C. F. (2007). Social Neuroeconomics: The Neural Circuitry of Social Preferences. Trends Cogn Sci.
• Harris, S., Sheth, S. A., & Cohen, M. S. (2007). Functional Neuroimaging of Belief, Disbelief, and Uncertainty. Annals of Neurology,
• Houde, O., & Tzourio-Mazoyer, N. (2003). Neural Foundations of Logical and Mathematical Cognition. Nature Reviews Neuroscience, 4(6), 507-514.
• Knutson, B., Rick, S., Wimmer, G. E., Prelec, D., & Loewenstein, G. (2007). Neural Predictors of Purchases. Neuron, 53(1), 147-156.
  
• Spitzer, M., Fischbacher, U., Herrnberger, B., Gron, G., & Fehr, E. (2007). The Neural Signature of Social Norm Compliance. Neuron, 56(1), 185-196.

NaturalRationality - a round-up

A round-up of the most popular posts on NaturalRationality:

1. The Philosophy of Social Cognition - I - The Other Minds
2. The Dictator Game and Radiohead.
3. New Paper in Neuroeconomics
4. Altruism: a Research Program
5. The Neuroeconomics of Social Norms: A Neo-Rationalist Account

enjoy !

A few papers on decision, the brain, and morality

A few papers worth reading:

A study on meta-ethics beleifs (how people see ethical claims):

• Goodwin, G. P., & Darley, J. M. (2008). The psychology of meta-ethics: exploring objectivism,Cognition, 106(3), 1339-1366.

Important for anyone interested in neuroscience: to a non-expert public, seeing a picture of a brain biases people to give more credibility to a piece of information:

• Deena Skolnick Weisberg, Frank C. Keil, Joshua Goodstein, Elizabeth Rawson, & Jeremy R. Gray. (2008, February 5). The seductive allure of neuroscience explanations.
  

Social cognition: it begins with goal recognition

• Rochat, M. J., Serra, E., Fadiga, L., & Gallese, V. (2008). The evolution of social cognition: goal familiarity shapes monkeys' action understanding, Curr Biol.

Jesse Prinz's new book (in a nutshell: morality is emotional and relative to a culture)

• The Emotional Construction of Morals, Jesse Prinz, Oxford UP

• The Palgrave Entry for "Field Experiments in Economics: (List, John A.& Reiley, David
  

Forthcoming in the new journal Neuroethics:

• Neuroethics and the Problem of Other Minds: Implications of Neuroscience for the Moral Status of Brain-Damaged Patients and Nonhuman Animals (Martha J. Farah)
  

A study on Chimpanzee barter behavior:

• Brosnan, S. F., Grady, M. F., Lambeth, S. P., Schapiro, S. J., & Beran, M. J. (2008). Chimpanzee autarky, PLoS ONE, 3(1), e1518.

A roundup of recent research

Happy new year everybody!

Here are a selection of recent papers you may enjoy:

• Neurons in the OFC encode economic value, not relative preference
• A Précis of Evolution in Four Dimensions, a book that proposes a new model of evolution
  
• Humans evolved rapidly in the last 40 000 years
  
• The cellular basis of decision-making in the archefish
  
• Rabbits value social contacts similarly to food
  
• Encouraging a belief in determinism increases cheating.
• Action understanding in novel situations is primarily mediated by an inferential interpretive system rather than the mirror system.

Two New Papers on Natural Rationality

Hardy-Vallée, B. (forthcoming). Decision-Making in the Economy of Nature: Information as Value. In G. Terzis & R. Arp (Eds.), Information and Living Systems: Essays in Philosophy of Biology. Cambridge, MA: MIT Press.

This chapter analyzes and discusses one of the most important uses of information in the biological world: decision-making. I will first present a fundamental principle introduced by Darwin, the idea of an “economy of nature,” by which decision-making can be understood. Following this principle, I then argue that biological decision-making should be construed as goal-oriented, value-based information processing. I propose a value-based account of neural information, where information is primarily economic and relative to goal achievement. If living beings (I focus here on animals) are biological decision-makers, we may expect that their behavior would be coherent with the pursuit of certain goals (either ultimate or instrumental) and that their behavioral control mechanisms would be endowed with goal-directed and valuation mechanisms. These expectations, I argue, are supported by behavioral ecology and decision neuroscience. Together, they provide a rich, biological account of decision-making that should be integrated in a wider concept of ‘natural rationality’.

Hardy-Vallee B. (submitted) Natural Rationality and the Psychology of Decision: Beyond bounded and ecological rationality

Decision-making is usually a secondary topic in psychology, relegated to the last chapters of textbooks. It pictures decision-making mostly as a deliberative task and rationality as a matter of idealization. This conception also suggests that psychology should either document human failures to comply with rational-choice standards (bounded rationality) or detail how mental mechanisms are ecologically rational (ecological rationality). This conception, I argue, runs into many problems: descriptive (section 2), conceptual (section 3) and normative (section 4). I suggest that psychology and philosophy need another—wider—conception of rationality, that goes beyond bounded and ecological rationality (section 5).

Natural Rationality on Facebook

Right here: http://apps.facebook.com/natural-rationality/
You can add it as an application to your profile.
Oh, and you can see me too.

Exploration, Exploitation and Rationality

A little introduction to what I consider to be the Mother of All Problems: the exploration-exploitation trade-off.

Let's firt draw a distinction between first- and second-order uncertainty. Knowing that a source of reward (or money, or food, etc.) will be rewarding in 70% of the occasions is uncertain knowledge because one does not know for sure what will be the next outcome (one can only know that there is a 70% probability that it is a reward). In some situations however, uncertainty can be radical, or second-order uncertainty: even the probabilities are unknown. Under radical uncertainty, cognitive agents must learn reward probabilities. Learners must, at the same time, explore their environment in order to gather information about its payoff structure and exploit this information to obtain reward. They face a deep problem—known as the exploration/exploitation tradeoff—because they cannot do both at the same time: you cannot explore all the time, you cannot exploit all the time, you must reduce exploration but cannot eliminate it. This tradeoff is usually modeled with the K-armed bandit problem.

Suppose an agent has n coins to spend in a slot machine with K arms (here K=2 and we will suppose that one arm is high-paying and the other low-paying, although the agent does not know that). The only way the agent has access to the arms’ rate of payment – and obtains reward – is by pulling them. Hence she must find an optimal tradeoff when spending its coins: trying another arm just to see how it pays or staying with the one who already paid? The goal is not only to maximize reward, but also to maximize reward while obtaining information about the arm’s rate. The process can be erroneous in two different ways: either the player can be victim of a false negative (a low-paying sequence of the high-paying arm) or false positive (a high-paying sequence paying of the low-paying paying arm).

To solve this problem, the optimal solution is to compute an index for every arm, updating this index according to the arm’s payoff and choosing the arm that has the greater index (Gittins, 1989). In the long run, this strategies amount to following decision theory after a learning phase. But as soon as switching from one arm to another has a cost, as Banks & Sundaram (1994) showed, the index strategies cannot converge towards an optimal solution. A huge literature in optimization theory, economics, management and machine learning addresses this problem (Kaelbling et al., 1996; Sundaram, 2003; Tackseung, 2004). Studies of humans or animals explicitly submitted to bandit problems, however, show that subjects tend to rely on the matching strategy (Estes, 1954). They match the probability of action with the probability of reward. In one study, for instance, (Meyer & Shi, 1995), subjects were required to select between two icons displayed on a computer screen; after each selection, a slider bar indicated the actual amount of reward obtained. The matching strategy predicted the subject’s behavior, and the same results hold for monkeys in a similar task (Bayer & Glimcher, 2005; Morris et al., 2006).

The important thing with this trade-off, is its lack of a priori solutions. Decision theory works well when we know the probabilities and the utilities, but what can we do when we don’t have them? We learn. This is the heart of natural rationality: crafting solutions—under radical uncertainty and non-stationary environments—for problems that may not have an optimal solution. Going from second- to first-order uncertainty.



See also:

• Hardy-Vallée, B. (2007). Artificial Life, Natural Rationality and Probability Matching. IEEE Symposium on Artificial Life, 2007, 123-129.
• Cohen, J. D., McClure, S. M., & Yu, A. J. (2007). Should I Stay or Should I Go? How the Human Brain Manages the Trade-Off between Exploitation and Exploration. Philosophical Transactions of the Royal Society B: Biological Sciences, 362(1481), 933-942.[Great paper on the subject]
  
  

References

• Banks, J. S., & Sundaram, R. K. (1994). Switching Costs and the Gittins Index. Econometrica: Journal of the Econometric Society, 62(3), 687-694.
• Bayer, H. M., & Glimcher, P. W. (2005). Midbrain Dopamine Neurons Encode a Quantitative Reward Prediction Error Signal. Neuron, 47(1), 129.
• Estes, W. K. (1954). Individual Behavior in Uncertain Situations: An Interpretation in Terms of Statistical Association Theory. In R. M. Thrall, C. H. Coombs & R. L. Davies (Eds.), Decision Processes (pp. 127-137). New York: Wiley.
• Gittins, J. C. (1989). Multi-Armed Bandit Allocation Indices. New York: Wiley.
• Kaelbling, L. P., Littman, M. L., & Moore, A. W. (1996). Reinforcement Learning: A Survey. Journal of Artificial Intelligence Research, 4, 237-285.
• Meyer, R. J., & Shi, Y. (1995). Sequential Choice under Ambiguity: Intuitive Solutions to the Armed-Bandit Problem. Management Science, 41(5), 817-834.
• Morris, G., Nevet, A., Arkadir, D., Vaadia, E., & Bergman, H. (2006). Midbrain Dopamine Neurons Encode Decisions for Future Action. Nat Neurosci, 9(8), 1057-1063.
• Sundaram, R. K. (2003). Generalized Bandit Problems: Working Paper, Stern School of Business.
• Tackseung, J. (2004). A Survey on the Bandit Problem with Switching Costs. De Economist, V152(4), 513-541.
• Yen, G., Yang, F., & Hickey, T. (2002). Coordination of Exploration and Exploitation in a Dynamic Environment. International Journal of Smart Engineering System Design, 4(3), 177-182.

A roundup of the most popular posts

According to the stats, the 5 mots popular posts on Natural Rationality are:

1. Strong reciprocity, altruism and egoism
2. What is Wrong with the Psychology of Decision-Making?
3. My brain has a politics of its own: neuropolitic musing on values and signal detection
4. Rational performance and behavioral ecology
5. Natural Rationality for Newbies
   

Enjoy!

Natural Rationality for Newbies

Decision-making, as I routinely argue in this blog, must be understood as entrenched in a richer theoretical framework: Darwin’s economy-of-nature. According to this principle, animals could be modeled as economic agents and their control systems could be modeled as economic devices. All living beings are thus deciders, strategists or traders in the economy of reproduction and survival.

When he suggested that nature is an economy, Darwin paved the way for a stronger interaction between biology and economics. One of the consequences of a bio-economic approach is that decision-making becomes an increasingly important topic. The usual, commonsense construal of decision-making suggests that it is inherently tied to human characteristics, language in particular. If that is the case, then talk of animal decisions is merely metaphorical. However, behavioral ecology showed that animals and human behavior is constrained by economic parameters and coherent with the economy-of-nature principle. Neuroeconomics suggest that the neural processing follow the same logic. Dopaminergic systems drive animals to achieve certain goals while affective mechanisms place goals and action in value spaces. These systems, although they were extensively studied in humans, are not peculiar to them: humans display a unique complexity of goals and values, but this complexity relies partly on neural systems shared with many other animals: the nucleus accumbens and the amygdala, for instance are common in mammals. Brainy animals evolved an economic decision-making organ that allows them to cope with complex situations. As Gintis remarks, the complexity and the metabolic cost of central nervous systems co-evolved in vertebrates, which suggests that despite their cost, brains are designed to make adaptive decision[i].

Hence decision-making should be analyzed similarly as, and occupies an intellectual niche analogous to, the concept of cooperation. Nowadays, the evolutionary foundations, neural substrates, psychological mechanisms, formal modeling and philosophical analyses of cooperation constitutes a coherent—although not unified—field of inquiry [ii]. The nature of prosocial behavior, from kin selection to animal cooperation to human morality, is best understood by adopting a naturalistic stances that highlights both the continuity of the phenomenon and the human specificity. Biological decision-making deserves the same eclecticism.

Talking about biological decision-making comes at a certain conceptual price. As many philosophers pointed out, whenever one is describing actions and decisions, one is also presupposing the rationality of the agent[iii]. When we say that agent A chose X, we suppose that A had reasons, preferences, and so on. The default assumption is that preferences and actions are coherent: the firsts caused the seconds, and the seconds are justified by the firsts. The rationality philosophers are referring to, however, is a complex cognitive faculty, that requires language and propositional attitudes such as beliefs and desires. When animals forage their environment, select preys, patches, or mates, no one presupposes that they entertain beliefs or desires. There is nonetheless a presupposition that “much of the structure of the internal mental operations that inform decisions can be viewed as the product of evolution and natural selection”.[iv] Thus, to a certain degree, the neuronal processes concerned with the use of information are effective and efficient, otherwise natural selection would have discarded them. I shall label these presuppositions, and the mechanisms it might reveal, “natural rationality”. Natural rationality is a possibility condition for the concept of biological decision-making and the economy-of-nature principle. One needs to presuppose that there is a natural excellence in the biosphere before studying decisions and constraints.

More than a logical prerequisite, natural rationality concerns the descriptive and normative properties of the mechanisms by which humans and other animals make decisions. Most concepts of rationality take only the descriptive or the normative side, and hence tend to describe cognitive/neuronal processes without concern for their optimality, or state ideal conditions for rational behavior. For instance, while classical economics considers rational-choice theory either as a normative theory or a useful fiction, proponents of bounded rationality or ecological rationality refuse to characterize decision-making as optimization.[v] Others advocate a strong division of labor between normative and descriptive project: Tversky and Kahneman, for instance, concluded from their studies of human bounded rationality that the normative and descriptive accounts of decision-making are two separate projects that “cannot be reconciled”[vi]

The perspective I suggest here is that we should expect an overlap between normative and descriptive theories, and that the existence of this overlap is warranted by natural selection. On the normative side, we should ask what procedures and mechanisms biological agents should follow in order to make effective and efficient decision given all their constraints in the economy of nature. On the descriptive side, we must assess whether a procedure succeeds in achieving goals or, conversely, what goals could a procedure aim at achieving. If there is no overlap between norms and facts, then either norms should be reconceptualized or facts should be scrutinized: it might be the case that norms are unrealistic or that we did not identify the right goal or value.

This accounts contrasts with philosophers (e.g. Dennett or Davidson) who construe rationality as an idealization and researchers who preach the elimination of this concept because of its idealized status (evolutionary psychologists, for instance[vii]). Thus, rationality can be conceived not as an a priori postulate in economy and philosophy, but as an empirical and multidisciplinary research program. Quine once said that “creatures inveterately wrong in their inductions have a pathetic but praiseworthy tendency to die out before reproducing their kind”[viii]. Whether it is true for inductions is still open to debate, but I suggest that it clearly applies to decisions.

Related posts

• Decision-making and the economy of nature
• The Economy of Nature: A Brief Introduction
• Rational performance and behavioral ecology

Notes and references

• [i] (Gintis, 2007, p. 3)
• [ii] See for instance how neuroscience, game theory, economic, philosophy, psychology and evolutionary theory interact in (E. Fehr & Fischbacher, 2002; Ernst Fehr & Fischbacher, 2003; Hauser, 2006; Penner et al., 2005).
• [iii] (Davidson, 1980; Dennett, 1987; Popper, 1994).
• [iv] (Real, 1994, p. 4)
• [v] (Chase et al., 1998; Gigerenzer, 2004; Selten, 2001)
• [vi] (Tversky & Kahneman, 1986, p. s272)
• [vii][vii] (Cosmides & Tooby, 1994)
• [viii] (Quine, 1969, p. 126)
  

References

• Chase, V. M., Hertwig, R., & Gigerenzer, G. (1998). Visions of Rationality. Trends in Cognitive Science, 2(6), 206-214.
• Cosmides, L., & Tooby, J. (1994). Better Than Rational: Evolutionary Psychology and the Invisible Hand. The American Economic Review, 84(2), 327-332.
• Davidson, D. (1980). Essays on Actions and Events. Oxford: Oxford University Press.
• Dennett, D. C. (1987). The Intentional Stance. Cambridge, Mass.: MIT Press.
• Fehr, E., & Fischbacher, U. (2002). Why Social Preferences Matter: The Impact of Non-Selfish Motives on Competition, Cooperation and Incentives. Economic Journal, 112, C1-C33.
• Fehr, E., & Fischbacher, U. (2003). The Nature of Human Altruism. Nature, 425(6960), 785-791.
• Gigerenzer, G. (2004). Fast and Frugal Heuristics: The Tools of Bounded Rationality. In D. Koehler & N. Harvey (Eds.), Blackwell Handbook of Judgment and Decision Making (pp. 62–88). Oxford: Blackwell.
• Gintis, H. (2007). A Framework for the Unification of the Behavioral Sciences. Behavioral and Brain Sciences, 30(01), 1-16.
• Hauser, M. D. (2006). Moral Minds : How Nature Designed Our Universal Sense of Right and Wrong. New York: Ecco.
• Penner, L. A., Dovidio, J. F., Piliavin, J. A., & Schroeder, D. A. (2005). Prosocial Behavior: Multilevel Perspectives. Annual Review of Psychology, 56(1), 365-392.
• Popper, K. R. (1994). Models, Instruments, and Truth: The Status of the Rationality Principle in the Social Sciences. In The Myth of the Framework. In Defence of Science and Rationality
• Quine, W. V. O. (1969). Ontological Relativity, and Other Essays. New York,: Columbia University Press.
• Real, L. A. (1994). Behavioral Mechanisms in Evolutionary Ecology: University of Chicago Press.
• Selten, R. (2001). What Is Bounded Rationality ? . In G. Gigerenzer & R. Selten (Eds.), Bounded Rationality: The Adaptive Toolbox (pp. 13-36). MIT Press: Cambridge, MA.
• Tversky, A., & Kahneman, D. (1986). Rational Choice and the Framing of Decisions. The Journal of Business, 59(4), S251-S278.

Rational performance and behavioral ecology

The oeconomy of nature is in this respect exactly of a piece with what it is upon many other occasions. With regard to all those ends which, upon account of their peculiar importance, may be regarded, if such an expression is allowable, as the favourite ends of nature, she has constantly in this manner not only endowed mankind with an appetite for the end which she proposes, but likewise with an appetite for the means by which alone this end can be brought about, for their own sakes, and independent of their tendency to produce it. Thus self-preservation, and the propagation of the species, are the great ends which Nature seems to have proposed in the formation of all animals. Mankind are endowed with a desire of those ends, and an aversion to the contrary; with a love of life, and a dread of dissolution; with a desire of the continuance and perpetuity of the species, and with an aversion to the thoughts of its intire extinction. But though we are in this manner endowed with a very strong desire of those ends, it has not been intrusted to the slow and uncertain determinations of our reason, to find out the proper means of bringing them about.

- Adam Smith, (1759)

One of the main goal of this blog (and the research that feeds it) is the development of a coherent and rigourous naturalized theory of rationality--or more precisely, a theory of natural rationality (I'll discuss the difference in another post). This theory-in-progress construes rationality as a natural feature of the biological world (and yes, fellows philosophers, I deal with the question of normativity, but another day). The big picture is the following: there is a distiction between rational competence and rational performance (as in linguistics): the competence is the set of mechanisms that make rational performance (= rational actions) possible. Neuroeconomics, as I see it, is the most promising research program that attempt to decipher the rational competence. An interesting possibility raised by these researches is that neural mechanisms involved in rational comptence may not be uniquely humans. In my phd thesis (in French, pdf here), I proposed that the whole vertebrate clade should be considered as the natural kind that implements the category "rational agents". I blogged a lot about neuroeconomics (competence), so today I'll talk about performance. How do animals behave rationally? In a basic, utility-maximizing sense: they optimize a utility function. While research in behavioral ecology showed that this hypothesis is justified, neureoconomics shows that it is more than an 'as-if' hypothesis or a useful fiction. So let's talk about behavioral ecology and rational performance.

Behavioral ecology models animals as economic agents that achieve ultimate goals (survival and reproduction) through instrumental ones (partner selection, food acquisition and consumption, etc.)[1]. Optimal foraging theory, for instance, represents foraging as a maximization of net caloric intake. With general principles derived from microeconomics, optimization theory and control theory, coupled with information about the physical constitution and ecological niche of the predator, it is possible to predict what kind of prey and patch an animal will favor, given certain costs such as search, identification, procurement, and handling costs. Optimal foraging theory (OFT), as their founders suggested, tries to determine “which patches a species would feed and which items would form its diet if the species acted in the most economical fashion”[2]. OFT models primarily animals as efficient goal-seekers and goal-achievers.
OFT thus incorporates agents, their choices, the currency to be maximized (most of the time a caloric gain) and a set of constraints. Most researches study where to forage (patch choice), what to forage (prey choice) and for how long (optimal time allocation). It is supposed that the individual animal make a series of decisions in order to solve a problem of sequential optimization. An animal looking for nutrients must maximize its caloric intake while taking into account those spent in seeking and capturing its prey; to this problem one must also add, among others, the frequency of prey encounter, the time devoted to research and the calories each prey type afford. All these parameters can be represented by a set of equations from which numerical methods such as dynamic programming allow biologists to derive algorithms that an optimal forager would implement in order to maximize the caloric intake. These algorithms are used afterward for the prediction of the behavior. Mathematically speaking, OFT is the translation of decision theory axioms—together with many auxiliary hypotheses—into tractable calories-maximization algorithms.
Economic models of animal behavior succeeded in explanation and prediction. It predicts for example how birds split their time between defending a territory and foraging[3], or between singing and foraging[4]. In their meta-analysis, Sih and Christensen[5] re-examined 134 foraging studies in laboratory and natural context, experimental or observational, and concluded that, although predictive success is not perfect, the predictivity of the theory is relatively high when preys are motionless (the prey can be a plant, seeds, honey, etc).
Interactive contexts are aptly modeled by game theory, mainly social foraging, fighting and predatory-preys relations[6]. For example, a model of Vickery et al[7] predicted that the co-occurrence of three social foraging strategies, producer (gathering nutrients) scrounger (stealing nutrients) and opportunist (switching between producer and scrounger) occurs only in the—very improbable—case where the losses opportunists would incur while foraging would be exactly equivalent to the profit of stealing. The model, however, predicts certain distributions of pairs of strategies that constitute evolutionary stable strategies (ESS), that is, a strategy that cannot be invaded by any competing alternative strategy. The proportion of food patch shared by scroungers, the size of the group and the degree of compatibility between the scrounger and producer strategy (i.e., if it is easy for the animal to perform both activities) determine the distribution of the strategies in a population, which was confirmed, inter alia, in birds (Lonchura punctulata)[8]. As predicted by the model, the producer strategy becomes less common when the cost of individual foraging increases.
Recently, behavioral ecologists found that animals could also be modeled as traders in biological markets. Obviously, biological markets do not have symbolic and conventional currencies systems, but in many interactions between animals institute trading structures. As soon as agents are able to provide commodities for mutual profit, the competition for obtaining commodities creates a higher bid. Animals seek and select partners according to the principle of supply and demand in interspecific mutualism, mate selection and intraspecific co-operation. An example of the last type is the cleaning market instituated by Hipposcarus harid fishes and cleaners-fishes Labroides dimidiatus. The “customers” (Hipposcarus) use the services of the cleaner to have its parasites removed, whereas the cleaners, occasionally, cheat and eat the healthy tissues of its customers. Since the cleaners offer a service that cannot be found elsewhere, they benefit from a certain economic advantage. A customer cannot choose to be exploited or not, whereas the cleaner chooses to cooperate or not (thus the payoffs are asymmetric). The customer—a predator fish that could eat the cleaner—abstains from consuming the cleaner in the majority of the cases, given the reciprocal advantage. Bshary and Schaffer[9] observed that cleaners spend more time with occasional customers than with regular ones and fight for them, since occasional customers are easier to exploit. All this makes perfect economic sense.
One could of course reformulate each of these results, and put the words decision or exchange between quotation marks to imply that they are mere façons de parler and not really decisions and exchanges, but instinctive behaviors preserved by natural selection. If this is the case, we should also put quotation marks when we talk about human beings: human behavioral ecology applies the same bio-economic logic with the same success to humans. Agents are modeled as optimal forager subjects to a multitude of constraints. Given available resources in the environment of a community, one can generates a model that predicts the optimal allocation of resources. These models are of course more complex than animal ones since they integrate social parameters like local habits, technology or economic structures. Models of human foraging where able for instance to explain differences in foraging style between tribes in the Amazonia, given the distance to be traversed and the technology used[10]. Food sharing, labor division between men and women, agricultural cultures and even Internet browsing (where the commodity is information) can be modeled by human behavioral ecology[11]. Hence even if foraging or trading behaviors are merely the execution of adaptations, the fact remains that their performance is best described as a decision-making process.


[1] (Krebs & Davies, 1997; Pianka, 2000)
[2] (MacArthur & Pianka, 1966, p. 603).
[3] (Kacelnik, Houston, & Krebs, 1981),
[4] (Thomas, 1999)
[5] (Sih & Christensen, 2001)
[6] (Hansen, 1986; Lima, 2002).(Dugatkin & Reeve, 1998)
[7] (Vickery, Giraldeau, Templeton, Kramer, & Chapman, 1991)
[8] (Mottley & Giraldeau, 2000)
[9] (Bshary & Schaffer, 2002)
[10] (Hames & Vickers, 1982)
[11] (Jochim, 1988; Kaplan, Hill, Hawkes, & Hurtado, 1984; Pirolli & Card, 1999)


Bshary, R., & Schaffer, D. (2002). Choosy reef fish select cleaner fish that provide high-quality service. Animal Behaviour, 63(3), 557.
Dugatkin, L. A., & Reeve, H. K. (1998). Game theory & animal behavior. New York ; Oxford: Oxford University Press.
Hames, R. B., & Vickers, W. T. (1982). Optimal Diet Breadth Theory as a Model to Explain Variability in Amazonian Hunting. American Ethnologist, 9(2, Economic and Ecological Processes in Society and Culture), 358-378.
Hansen, A. J. (1986). Fighting Behavior in Bald Eagles: A Test of Game Theory. Ecology, 67(3), 787-797.
Jochim, M. A. (1988). Optimal Foraging and the Division of Labor. American Anthropologist, 90(1), 130-136.
Kacelnik, A., Houston, A. I., & Krebs, J. R. (1981). Optimal foraging and territorial defence in the Great Tit (Parus major). Behavioral Ecology and Sociobiology, 8(1), 35.
Kaplan, H., Hill, K., Hawkes, K., & Hurtado, A. (1984). Food Sharing Among Ache Hunter-Gatherers of Eastern Paraguay. Current Anthropology, 25(1), 113-115.
Krebs, J. R., & Davies, N. B. (1997). Behavioural ecology : an evolutionary approach (4th ed.). Oxford, England ; Malden, MA: Blackwell Science.
Lima, S. L. (2002). Putting predators back into behavioral predator-prey interactions. Trends in Ecology & Evolution, 17(2), 70.
MacArthur, R. H., & Pianka, E. R. (1966). On optimal use of a patchy environment. American Naturalist(100), 603-609.
Mottley, K., & Giraldeau, L. A. (2000). Experimental evidence that group foragers can converge on predicted producer-scrounger equilibria. Anim Behav, 60(3), 341-350.
Pianka, E. R. (2000). Evolutionary ecology (6th ed.). San Francisco, Calif.: Benjamin Cummings.
Pirolli, P., & Card, S. (1999). Information Foraging. Psychological Review, 106(4), 643.
Sih, A., & Christensen, B. (2001). Optimal diet theory: when does it work, and when and why does it fail? Animal Behaviour, 61(2), 379.
Smith, A. ([1759] 2002). The theory of moral sentiments. Cambridge, U.K. ; New York: Cambridge University Press.
Thomas, R. J. (1999). Two tests of a stochastic dynamic programming model of daily singing routines in birds. Anim Behav, 57(2), 277-284.
Vickery, W. L., Giraldeau, L.-A., Templeton, J. J., Kramer, D. L., & Chapman, C. A. (1991). Producers, Scroungers, and Group Foraging. American Naturalist, 137(6), 847-863.

A round-up of recent papers on natural rationality

See other news in my Shared Items here or subscribe to the feed

• The neuroscience of trust: amygdala activity increases as perceived trustworthiness decreases (JOCN)
• The social appropriateness of jokes is encoded through VMPFC and hippocampus interaction (JOCN)
  
• Delusional beliefs may be causes by a malfunctioning of prediction-error-related signal in rPFC (Brain)
  
• Primatologist Frans de Waal criticised the spindoctoring of science for political purposes (data about chimpanzee and bononbos) (eSekeptic)
  
• The neural basis of risky decision-making in a blackjack task. (Neuroreport)
• Morality and psychopathy correspond to well- and mal-functioning of amygdala and ventromedial prefrontal cortex interaction (Trends in cognitive science)
  

A basic mode of behavior: a review of reinforcement learning, from a computational and biological point of view.

The Journal Frontiers of Interdisciplinary Research in the Life Sciences (HFSP Publishing) made its first issue freely available online. The Journal specializes in "innovative interdisciplinary research at the interface between biology and the physical sciences." An excellent paper (complete, clear, exhaustive) by Kenji Doya presents a state-of-the-art review of reinforcement learning, both as a computational theory (the procedures) and a biological mechanism (neural activity). Exactly what the title announces: Reinforcement learning: Computational theory and biological mechanisms. The paper covers research in neuroscience, AI, computer science, robotics, neuroeconomics, psychology. See this nice schema of reinforcement learning in the brain:

(From the paper:) A schematic model of implementation of reinforcement learning in the cortico-basal ganglia circuit (Doya, 1999, 2000). Based on the state representation in the cortex, the striatum learns state and action value functions. The state value coding striatal neurons project to dopamine neurons, which sends the TD signal back to the striatum. The outputs of action value coding striatal neurons channel through the pallidum and the thalamus, where stochastic action selection may be realized

This stuff is exactly what a theory of natural rationality (and economics tout court): plausible, tractable, and real computational mechanism grounded in neurobiology. As Selten once said, speaking of reinforcement learning:

a theory of bounded rationality cannot avoid this basic mode of behavior (Selten, 2001, p. 16)

References

• Kenji, D. (2007). Reinforcement learning: Computational theory and biological mechanisms. HFSP Journal, 1(1), 30-40.
• Selten, R. (2001). What is bounded rationality ? . In G. Gigerenzer & R. Selten (Eds.), Bounded Rationality: The Adaptive Toolbox (pp. 13-36). MIT Press: Cambridge, MA.
  

Decision-Making: A Neuroeconomic Perspective

I put a new paper on my homepage :

Decision-Making: A Neuroeconomic Perspective

Here is the abstract:

This article introduces and discusses from a philosophical point of view the nascent field of neuroeconomics, which is the study of neural mechanisms involved in decision-making and their economic significance. Following a survey of the ways in which decision-making is usually construed in philosophy, economics and psychology, I review many important findings in neuroeconomics to show that they suggest a revised picture of decision-making and ourselves as choosing agents. Finally, I outline a neuroeconomic account of irrationality.

Hardy-Vallée, B. (forthcoming). Decision-making: a neuroeconomic perspective. Philosophy Compass. [PDF]

This paper is the first in my philosophical exploration of neuroeconomics, and I would gladly welcome your comments and suggestions for subsequent research. Email me at benoithv@gmail.com.

[Book review] Rediscovering Empathy. Agency, Folk Psychology, and the Human Sciences

Rediscovering Empathy Agency, Folk Psychology, and the Human Sciences by Karsten R. Stueber MIT Press, 2006 Review by Benoit Hardy-Vallée, Ph.D. on Jul 3rd 2007 Volume: 11, Number: 27

Many philosophers and cognitive scientists are now familiar with a traditional debate between two accounts of folk-psychology (the intuitive framework of beliefs, desires and intentions we use everyday to understand each others). According to the first one--the theory-theory account, or "information-rich"--we apply a psychological theory to other's actions and infer, on that theoretical basis, the reasons that motivate their actions. To the contrary, the simulation account, or "information-poor", holds that folk-psychology is essentially imitative and imaginative: we use ourselves as simulators of others agents' mind in order to gain information about their reasons to act. The debate was particularly vigorous in the 90's (philosophers, psychologists, primatologists and cognitive scientists participated) but seemed to vanish in the recent years. In his new book, Rediscovering Empathy: Agency, Folk Psychology, and the Human Sciences, Karsten Stueber is not only putting forth a welcome revival of the contention, but is also deepening it.

Beside the mechanistic questions of how we process information in order to explain and predict actions, the disagreement between both parties is also epistemological. Simulationists do not only argue that we use different skills, but also that there is something radically different in the way we understand other agents (vs. the way we understand the rest of the physical and world). When we interpret other persons, we conceive them as minded creatures that have a first-person perspective, just like we do. When we interact with, or think about, other physical objects, we don't use our imagination to simulate their subjective point of view, since we don't take them to have one. Thus, for the simulationists, agents and objects are structurally different. There are no common inferential mechanisms that apply to both.

In his book, Stueber makes a strong case in favor of simulation, or more generally, empathy. His thesis is that empathy has a central epistemic role: it is the default mode of interpersonal understanding. We first and foremost comprehend actions by putting ourselves in someone else's shoes, not by relying on a psychological theory of human cognition. Cognizing other agents is essentially an 'engaged' task, not a 'detached' one: we use ourself--our emotions, sensations, and thoughts--as mindreading tools, not an external device such as theory. Stueber compares mindreading and judging whether someone is the same height as yourself. You can either use an external, neutral standard--a measuring tape for instance--or use yourself as a standard: see if your head and hers is at same level. In this case, we use a subjective, non-neutral and egocentric point of view.

Stueber draws a distinction between what he calls basic and reenactive empathy. The first one is a quasi-perceptual mechanism (implemented in so-called 'mirror neurons'): when we see someone scared, we can easily sense her feeling. We see that someone is scared, but not why. Understanding why requires a more complex kind of empathy or reenactment. This second type of empathy, realized through a deliberative process, allows us to understand the reasons of actions. Since thought is essentially contextual and indexical, understanding someone else's thoughts requires that we see others' thoughts as thoughts that, had they been ours in the same context, would give use reason to act. Thus it is by inner imitation that we really grasp others' intentions, not by theoretical deduction.

It is impossible, however, to do so without viewing each other as rational beings. Rational agency is a condition for reenactive empathy: when we take others to be normatively assessable, we can reconstruct the thought processes that govern their actions. By 'rational agency', Stueber does not imply that humans are good logicians or rational-choice theorists. A rational creature, he argues, is a creature whose assertions and actions are motivated by reasons, and whose reasons can be evaluated in the light of normative theories of rationality. Thus, all empirical studies of showing that humans do not comply with logic and rational-choice theory do not undermine the role of rational agency in folk-psychological interpretation. Hence empathy is inherently based on a rationality assumption, what other philosophers such as Davidson called the principle of charity: interpreting others' beliefs as coherent.

Having established empathy as the central mindreading device and rejected theory-theory and other detached accounts, Stueber goes on to claim that empathy also has a normative role: it justifies our beliefs-desires attributions. Here again, the author uses a vivid analogy. We can justify a prediction that it will rain tomorrow only by using an information-poor background: the barometer says so, and the barometer is a reliable tool. Thus the prediction is inductively justified. Similarly, as long as we are in the domain of psychological interpretation, empathy is a reliable predictive tool that doesn't require a rich theoretical background. Of course, as many objected, empathy might be fallible, since it can be influenced by cultural and social background. Yet empathetic reenactement is still, Stueber contends, the principal mindreading strategy. It is fallible but it can be supplemented with auxiliary information.

Rediscovering Empathy is not just another book about folk-psychology. It is a systematic enquiry into the structure and function of mindreading that goes beyond the traditional exposition of recent cognitive theories. As Stueber shows, the debate between the engaged and detached conception of interpretation is not new and has roots in 19^th century discussions of hermeneutic Verstehen (understanding) and aesthetic Einfühlung (empathy). The nature and function of empathy is relevant for a diverse array of empirical and theoretical inquiries: beyond cognitive science and philosophy of mind, the debate concerning the nature of folk-psychological understanding impacts upon foundational debates in hermeneutics, aesthetics, anthropology, neuroscience, philosophy of language and philosophy of social science (mainly philosophy of history). Interpreting other agents, artifacts, texts, historical events or different cultures requires some kind of mechanisms that reliably indicates why individuals do what they do.

The greatest strength of this book is its ability to guide the reader through many important philosophical and scientific debates: the simulation vs. theory-theory debate, the rationality debate, the significance of mirror neurons and the nature of historical explanation, principally. In every case, the author uses an acute terminology and provides a clear presentation of competing theories, their empirical basis, their conceptual significance and their position in the history of thought. He exposes complex problems but never looses the reader (except in the last chapters, where the arguments is less clear). One notes also the unity and coherence of the book. The only problem with the book is that the author takes certain claims (e.g., the contextuality of thought) to be purely 'conceptual': he accepts them without much justification and thus it seems more dogmatic than conceptual.

This book will be of interest for any scholars interested in interpretation, generally speaking, but might be more accessible for philosophers of mind and social science. Cognitive scientists, social psychologists and social scientists will also found many discussions in the book relevant for their field.

Note: The introduction of the book can be freely downloaded on the publisher's website:

http://mitpress.mit.edu/books/chapters/026219550Xintro1.pdf

Social Neuroeconomics: Strong Reciprocity or “Hot Logic”?

The presentation of a talk I recently gave in Montreal at Cognitio 2007.




The experimental study of economic exchange behavior revealed many discrepancies between normative theory of strategic rationality (game theory) and actual behavior. In many games games where defection and competition is expected by game theory, subjects robustly display cooperative behavior. In the ultimatum game, for instance, a ‘proposer’ makes an offer to a ‘responder’ that can either accept or refuse the offer; if the responder refuses, both players get nothing. The rational outcome is a minimal offer by the first player and an unconditional acceptance of the offer by the second. In fact, proposers make ‘fair’ offers, about 50% of the amount, responders tend to accept these offers and reject most of the ‘unfair’ offers (less than 20%;Oosterbeek et al., 2004). Cooperative and prosocial behavior is also observed in similar games, e.g. the trust game and the prisoner’s dilemma (Camerer, 2003). Neuroeconomics, the study of the neural mechanisms of decision-making (Glimcher, 2003), also showed that subjects seems to entertain prosocial preferences. Brain scans of people playing the ultimatum game indicates that unfair offers trigger, in the responders’ brain, a ‘moral disgust’: the anterior insula, an area involved in disgust and other negative emotional responses, is more active when unfair offers are proposed (Sanfey et al., 2003). In the prisoner’s dilemma and the trust game, similar activations have been found: cooperation and punishment of unfair players elicit positive affective emotions, while unfairness elicit negative one (de Quervain et al., 2004; Rilling et al., 2002).
The received view of these behavioral and neural data is that human beings are endowed with genuinely altruistic cognitive mechanisms, a view now labelled “Strong Reciprocity” (SR). According to SR, an innate propensity for altruistic punishment and altruistic rewarding makes us averse to inequity (Fehr & Rockenbach, 2004). In this talk, I argue that this moral optimism is far-fetched. Yes, the ‘cold logic’ model of rationality is not an accurate description of our decision-making mechanisms, but the SR model, I shall argue, relies on unwarranted assumptions. I present another model–the ‘hot logic’ approach–according to which human agents are selfish agents adapted to trade, exchange and partner selection in biological markets (Noë et al., 2001). Cognitive mechanisms of decision-making aims primarily at maximizing positive outcomes and minimizing negative ones. This initial hedonism is gradually modulated by social norms, by which agents learn how to maximise their utility given the norms. The ‘hot logic’ approach provide a simpler explanation of cooperation and fairness: subjects make ‘fair’ offers in the ultimatum game because they know their offer would be rejected otherwise. Responders affective reaction to ‘unfair offers’ is in fact a reaction to the loss of an expected monetary gain: they anticipated that the proposer would comply with social norms. This claim is supported by other imaging studies showing that loss of money can be aversive, and that actual and counterfactual utility recruit the same neural resources (Delgado et al., 2006; Montague et al., 2006). This approach explains why subjects make lower offers in the dictator game (an ultimatum game in which the responder make an offer and the responder's role is entirely passive) than in the ultimatum, why, when using a computer displaying eyespots, almost twice as many participants transfer money in the dictator (Haley & Fessler, 2005), and why attractive people are offered more in the ultimatum (Solnick & Schweitzer, 1999). In every case, agents seek to maximize a complex hedonic utility function, where the reward and the losses can be monetary, emotional or social (reputation, acceptance, etc.). SR is thus seen as cooperative habits that are not repaid (Burnham & Johnson, 2005).

Natural Rationality papers: A round-up

• Socioeconomic gradients predict individual differences in neurocognitive abilities
• Young children (before 4 years-olds) do not seem to be able to connect their present decisions to their future selves
• Altruism is predominantly paternalistic: subjects prefer to give what is good for someone (from the giver's point of view) instead of what someone would like
• Animal personalities are life-history strategies.
  
• Expertise exists everywhere in nature, from honeybees to humans
  
• Monkeys have a "a crude capacity for probabilistic inference"