Tobacco Documents Online

li -! Neuroscience and Biobehavioral Reviews, Vol. 21, No. 3, pp. 341-359, 1997 Copyright @ 1997 Elsevier Science Ltd Printed in Great Britain. All rights reserved 0149-7634/97 $32.00 + .0C PIh S0149-7634(96)00017-6 Behavioral Functions of Nucleus Accumbens Dopamine: Empirical and Conceptual Problems with the Anhedonia Hypothesis J. D. SALAMONE, X M. S. COUSINS AND B. J. SNYDER Department of Psychology, University of Connecticut, Storrs, CT 06269-1070 USA SALAMONE, J. D., M. S. COUSINS AND B. J. SNYDER. Behavioral functions of nucleus accumbens dopamine: empirical and conceptualproblems with the anhedonia hypothesis. NEUROSCI BIOBEHAV REV. 21(3) 341-359. 1997.--Nucleus aceumbens (DA) has been implicated in a number of different behavioral functions, but most commonly it is said to be involved in "reward" or "reinforcement". In the present article, the putative reinforcement functions of aceumbens DA are summarized in a manner described as the "General Anbedonia Model". According to this model, the DA innervation of the nucleus accumbens is conceived of as a crucial link in the "reward system", which evolved to mediate the reinforcing effects of natural stimuli such as food. The reward system is said to be activated by natural reinforcing stimuli, and this activation mediates the reinforcing effects of these natural stimuli. According to this view, other stimuli such as brain stimulation and drugs can activate this system, which leads to these stimuli being reinforcing as well. Interference with DA systems is said to blunt the reinforcing effects of these rewarding stimuli, leading to "extinction". This general model of the behavioral functions of accumbens DA is utilized widely as a theoretical framework for integrating research findings. Nevertheless, there are several difficulties with the General Anhedonia Model. Several studies have observed substantial differences between the effects of extinction and the effects of DA antagonism or aecumbens DA depletions. Studies involving aversive conditions indicate that DA antagonists and accumbens DA depletions can interfere with avoidance behavior, and also have demonstrated that accumbens DA release is increased by stressful or aversive stimuli. Although accumbens DA is important for drug abuse phenomena, particularly stimulant self-administration, studies that involve other reinforcers are more problematic. A large body of evidence indicates that low doses of dopamine antagonists, or depletions of aceumbens DA, do not impair fundamental aspects of food motivation such as chow consumption and simple instrumental responses for food. This is particularly important, in view of the fact that many behavioral researchers consider the regulation of food motivation to be a fundamental aspect of food reinforcement. Finally, studies employing cost/benefit analyses are reviewed, and in these studies considerable evidence indicates that accumbens DA is involved in the allocation of responses in relation to various reinforcers. Nucleus aceumbens DA participates in the function of enabling organisms to overcome response costs, or obstacles, in order to obtain access to stimuli such as food. In summary, nucleus accumbens DA is not seen as directly mediating food reinforcement, but instead is seen as a higher order sensofimotor integrator that is involved in modulating response output in relation to motivational factors and response constraints. Interfering with accumbens DA appears to partially dissociate the process of primary reinforcement from processes regulating instrumental response initiation, maintenance and selection. © 1997 Elsevier Science Ltd. Dopamine Nucleus accumbens Reinforcement Reward "Motivation Motor Behavioral economics 1. INTRODUCTION IT has been suggested that dopamine (DA), particularly in the nucleus accumbens, is critically involved in the process of reinforcement. Indeed, this idea is currently one of the most popular in all of neuroscience; one can scarcely open a textbook, or peruse the pages of a jour- nal, without encountering this hypothesis. The general outline of this hypothesis typically proceeds as follows: it is thought that DA directly mediates the rewarding effects of natural stimuli such as food, water or sex. In turn, this reward system is activated by other reinforcing tTo whom correspondence should be addressed stimuli such as brairi stimulation or drugs of abuse. The "reward" hypothesis of DA is widely applied to explain a number of results, and is probably as broadly accepted and widely cited as the dual hypothalamic model of motivation was in the 1960s. The purpose of the present review is to offer a critical examination of the hypothesis that DA in nucleus accumbens directly mediates the reinforcing properties of natural stimuli such as food. In order to do so, various research findings and concep- tual issues will be examined. Finally, alternatives to the anhedonia hypothesis will be explored. 341 THIS ARTICLE IS FOR INDIVIDUAL USE ONLY AND HAY NOT BE FURTHER REPRODUCED OR STORED ELEOTRONICALLY WITHOUT WRITTEN PERHISSION FRON THE COPYRIGHT HOLDER.~'~ UNAUTHORIZED REPRODUCTION HAY RESULT IN FINANCIAL ,A~ OTHER PENALTIES.

342 2. THE ANHEDONIA HYPOTHESIS: SUPPORTING EVIDENCE One of the major functions of a scientific hypothesis is to stimulate research. In this regard, the DA/reinforcement hypothesis has been quite successful. Over the last two dec- ades, hundreds of articles have been published that deal directly or indirectly with the hypothesized involvement of DA systems in reinforcement processes. For the present review, it is useful to begin by examining those results that led to the initial proposal of the DA/reinforcement hypoth- esis, and also to outline those results that have continued to be offered as support for this hypothesis. 2.1. Historical development The 1970s was a period of rapid development in neu- roscience, and particular emphasis was being placed on sin- dies of the behavioral functions of catecholamines: Intracranial self-stimulation (ICSS) was one of the beha- viors that was being widely investigated; at that time, it was hoped that this phenomenon could yield important insights into the brain mechanisms underlying reinforce- ment. Several studies indicated that DA systems were criti- cally involved in ICSS (48,76,84,135). Fouriezos and Wise (76) observed that pimozide-treated rats showed near normal responding during the beginning of ICSS lever press test sessions, but that there was a within-session decline in responding across the session. It was noted that this pattern resembled the effects of extinction, and there- fore it was suggested that DA antagonism was interfering with the basic process of reinforcement. In reviewing the literature on DA and ICSS, Wise (247) suggested that interference with DA systems could be blunting the euphoria produced by ICSS and drugs of abuse. Moreover, it was suggested that additional research should study the effects of DA antagonists on behaviors supported by natural reinforcers such as food. Several studies by Wise and his colleagues were con- ducted to investigate the effects of pimozide on food-rein- forced instrumental responding. Wise et al. (253,254) observed that the effects of pimozide resembled those of extinction under several different conditions. Lever press responding for food or saccharin was suppressed by pimo- zide in a manner that was characterized by a within-session decline in responding. Food-reinforced running in an alley- way showed a gradual slowing over successive trials in pimozide-treated rats. These studies were interpreted to mean that DA antagonism blunted the reinforcing effects of food, which led to "extinction". It should, also be emphasized that these effects on food-reinforced responses were not viewed in isolation, and instead were interpreted in the context of the effects of DA antagonists on amphetamine self-administration and ICSS (253,254). Thus, the general hypothesis was offered that DA critically mediated the reinforcement produced by a wide variety of stimuli, includ- ing food, water, sex, drugs and brain stimulation. There were two important features of this early genera- tion of research that should be emphasized; both of these issues were evident in the ICSS literature and then extended into the arena of the DA/reinforcement hypothesis. First, the hypothesized effect of DA antagonism was referred to as "anhedonia". The use of this term indicated that the effects of DA antagonism were being explained in terms SALAMONE, COUSINS AND SNYDER of a blunting of the emotional effects of reinforcing stimuli. According to Wise et al. (253), neuroleptics were hypothe- sized to "take the pleasure out of normally rewarding brain stimulation, take the euphoria out of normally rewarding amphetamine, and take the 'goodness' out of normally rewarding food". The second major characteristic of the early generation of research on the DAJreinforcement hypothesis was that the effects of DA antagonism on "reinforcement" were conceived of as being completely distinct from any motor effects of these drugs. Concerns about the possible motor effects of DA antagonists grew naturally out of the extensive literature implicating basal ganglia DA in the control of movement. Of course, propo- nents of the anhedonia hypothesis maintained that the same animal could have both "reward" and "motor" effects resulting from administration of DA antagonists (e.g. Wise, 249). Nevertheless, they considered that these effects fall into completely distinct classes, and thus the dichotomy between the "reward" and "motor" effects of DA antagonism was fully drawn. 2.2. Additional evidence supporting the ahhedonia hypothesis As noted above, it was reported that systemic administra- tion of DA antagonists produced effects that were thought to resemble extinction. Several additional lines of evidence are usually cited as support for the DAJreinforcement hypoth- esis. Naturally occurring reinforcers are thought to increase DA release, particularly in nucleus accumbens (94,196). DA antagonists have been shown to reduce consumption of sweet rewards, such as sucrose solutions (103,204- 206,211). Several drugs of abuse have been shown to increase extracellular DA; this appears to be true not only of stimulants but also of drugs from other categories (31,55). Evidence has indicated that DA systems are particularly important for drug-related reinforcement phenomena such as place preference and drug self-administration (22,23,169,179,217). Because much of the work that led to the inception of the anhedonia hypothesis involved the use of systemic neuro- leptic administration, particular DA terminal regions were not initially identified as critical loci for the site at which DA systems mediated reinforcement. Yet, as research in this area developed it became common to identify the nucleus accumbens as the specific area in which DA mediated "reward". The nucleus accumbens is the site at which natural and drug reinforcers are thought to preferentially activate DA systems (31,55,94,100,144). The mesolimbic DA system, which originates in ventral tegmental area (VTA) and termi- nates in nucleus accumbens, has been implicated in drug self- administration. In some ways, the distinction between meso- limbic and neostriatal DA is thought to reflect the dichotomy between reward and motor function. Thus. nucleus accum- bens is thought to be closely related to the emotional processes in which the limbic system participates, while the neostriatum is thought to be involved in motor functions that are more traditionally ascribed to the basal ganglia. 2.3. The general anhedonia model To summarize, it has been suggested that nucleus accum- bens is a critical locus for mediating the reinforcing effects

BEHAVIORAL FUNCTIONS OF NUCLEUS ACCUMBENS DOPAMINE 343 al of stimuli. The DA innervation of the nucleus accumbens is conceived of as a crucial link in the "reward system", which evolved to mediate the reinforcing effects of stimuli such as food. This system is thought to be activated by natural reinforcing stimuli, and this activation mediates the reinforcing effects of natural stimuli. According to this view, other stimuli such as brain stimulation and drugs can activate this system, which leads to these stimuli being reinforcing as well. Interference with DA systems, particu- larly in nucleus accumbens, blunts the reinforcing effects of these rewarding stimuli, leading to "extinction". This gen- eral model of the behavioral functions of accumbens DA is utilized widely as a theoretical framework for research and a pedagogical device (e.g. (13,27,87,248)). Because of the broad nature of this model, and the variety of reinforcing conditions it is meant to explain, it will be referred to as the "General Anhedonia Model". The General Anhedonia Model has been enormously sue-' cessful in that it has offered researchers a framework for linking together various areas of research. Perhaps the great- est influence of this model has been in the field of drug self- administration and abuse. It is evident that the General Anhedonia Model is the most influential wide-ranging model for dealing with drug abuse. This model is used to explain stimulant self-administration and place preference in animals, abuse of a number of stimulant and non-stimu- lant drugs in humans, and even genetic factors leading to drug abuse. Yet, despite its ubiquity and utility, the General Anhedonia Model is not universally supported. The set of hypotheses that form the structure of this model are still fiercely debated. It is possible that some of the tenets of the model are inaccurate or overly simplistic, and that the model may have value for some areas (e.g. stimulant abuse) but not others (e.g. food reinforcement). The purpose of this review is to deconstruct the General Anhedonia Model and consider the limitation of each separate component. In par- ticular, this review will focus on the hypothesis that DA in nucleus accumbens directly mediates food reinforcement. 3. EMPIRICAL AND CONCEPTUAL PROBLEMS wrI'H THE ANHEDONIA HYPOTHESIS As noted above, a major function of a hypothesis is to stimulate research. In fact, not all of the research stimulated by the General Anhedonia Model has provided support for that model. Several research findings cause specific pro- blems for aspects of the anhedonia model. Moreover, there are conceptfial problems inherent in a discussion of brain mechanisms of reinforcement that are not often exam- ined in sufficient detail in the anhedonia literature. 3.1. Lack of similarity between interference with DA systems and the effects of extinction Although it often is cited that DA antagonists produce effects similar to extinction, it has been argued that these similarities are superficial in nature, and that under a broad range of conditions interference with DA systems does not in fact produce effects that closely resemble extinction. Several studies providing a detailed behavioral analysis have shown that there are substantial differences between the effects of extinction and systemic administration of DA antagonists (10, 62, 65, 71, 72, 89, 90, 148, 168, 185, 213, 228,230,244). Injections of flupethixol into the nucleus accumbens failed to produce an extinction-like decline in responding (15). Nucleus accumbens DA depletions also failed to produce effects that were similar to extinction if rats were responding on a continuous reinforcement (CRF) schedule (149,195). Nucleus accumbens DA depletions affect CRF or fixed ratio (FR) 5 responding by producing a slow, steady rate of responding throughout the session rather than a within session decline in responding (149,195,196). Moreover, extinction of CRF responding produces an extinction "burst", of which one manifestation is an "increase" in the proportion of responses with high local rates (195). In contrast, accumbens DA depletions produce the opposite effect, which is a "decrease" in the relative number of high rate responses that manifests itseif as an overall slowing of the local rate of responding (I95). Several studies have indicated that changing the kinetic requirements of an instrumental response can alter the extent to which an "extinction" effect is produced. Doses of DA antagonists that "extinguish" CRF lever pressing fail to produce extinction of differential-reinforcement of low-rate lever pressing (148), nose-poking for electrical stimulation (63), or simple instrumental behaviors such as being in proximity to a food dish (185). There is a conceptual problem with the extinction phe- nomenon as well. It has been stated that a within session decline in responding is prima facie evidence that a motor deficit cannot be in operation. According to Wise et al. (254) the motor hypothesis "demands that there be no period within a pimozide test when response rates are normal", and also "the fact that normal or near normal responding is not seen during the latter part of the sessions must be attributed to some other cause than mere response difficulties". In fact, this is the fut~damental assumption that underlies the entire notion that the within-session decline in responding represents an effect similar to extinction. Yet it should be emphasized that the validity of this assertion has rarely been seriously evaluated. Is it true that if responding is higher in the beginning of a test session, and declines thereafter, then it is impossible to explain this effect by discussing any aspect of motor function? For several years, it has been suggested by several different authors that the "mainte- nance" of movement is affected by DA-related manipula- tions (8,82,186). Fowler and his colleagues have studied the effects of neuroleptics on the duration of individual lever pressing responses, and have observed that neuroleptics increase (i.e. slow) lever press duration in a way that differs substantially from extinction (71,72,79). They also have noted that haloperidol produces within-session increases in the duration of lever press responses (133). Taking all this into account, it is possible that some of the behavioral changes produced by interference with DA can manifest themselves as response maintenance deficits or progressive motor dysfunctions rather than impairments in emotional processes (77,186). 3.2. Accunlbens DA is involved in aversive as well as appetitive conditions There is an extensive literature on the involvement of DA in processes that involve aversive or stressful conditions. Although this research is rarely, if ever, cited by advocates

344 of the anhedonia hypothesis, it should be emphasized that two recent reviews have thoroughly discussed these find- ings (18,190). A brief review of this area is useful for the present discussion. Perhaps the most important point to make is that accumbens DA is not selectively involved in appetitive behavioral processes. Also, there are substantial similarities between the characteristics of dopaminergic involvement in appetitively and aversively motivated behavior. Just as DA antagonists reliably reduce positively reinforced instrumental responding, it also is widely reported that neuroleptics impair avoidance responding (3,5,14,20,38,39,52,116,164,165,172,238). Nucleus accum- bens DA depletions and intra-accumbens injections of the DA antagonist sulpiride have been shown to impair avoid- ance responding (152,210,234). Although it is frequently cited that DA antagonists interfere with place preference, it should be emphasized that DA antagonists also impair place aversion. Di Scala and Sandner (56) reported that halo- peridol blocked the place aversion produced by the anxio- genic drug FG 7150. SCH 23390 and metoclopramide reduced both amphetamine-induced place preference and SKF-38393-induced place aversion (101). SCH 23390 was shown to block the place aversions produced by naloxone, picrotoxin and phencyclidine (2). Several other features of the effects of neuroleptic drugs on instrumental behavior, such as the within-session decline in responding and the relative preservation of discrimination performance, also have been demonstrated to occur with neuroleptic-treated rats responding on avoidance tasks (9,42,201). As noted above and discussed in greater detail below, appetitive stimulus conditions can be accompanied by increases in accumbens DA release as measured by techni- ques such as voltammetry or microdialysis. At this point, it is important to stress that aversive conditions also are accompanied by increases in accumbens DA release or metabolism. DA turnover or metabolite levels in accumbens or VTA tissue samples have been shown to increase in response to aversive stimuli (53,54,58,59,70,181). Voltam- metry and dialysis studies have shown that accumbens DA release or metabolism can be increased in response to tail shock (1), tail pinch (50) foot shock (212), restraint stress (108,109), forced exercise (5 I), and anxiogenic beta carbo- line drugs (51,150,202). Young et al. (260) reported that DA release in nucleus accumbens increased during the presenta- tion of a stimulus that had been paired with footshock. McCullough et al. (152) observed that lever pressing to avoid shock was accompanied by increases in accumbens DA release. Taking all these results into account, it seems obvious that enhanced accumbens DA release cannot simply be considered as a selective marker for subjective hedonia. 3.3. Accumbens DA release and neuronal activity do not covary specifically with the presentation of positive reinforcers Several studies have examined the dynamic activity of accumbens DA during the performance of positively rein- forced instrumental behavior. Generally, it has been observed that accumbens DA release is increased during performance of food-reinforced lever pressin~ (94,113,123,149,191,194). However, several lines of evi- dence suggest that it is not the presentation of the reinforcer SALAMONE, COUSINS AND SNYDER per se that instigates accumbens DA release or VTA neuronal activity. The effects of presentation of large quantities of food to food deprived animals are somewhat equivocal, with some studies reporting increases in accum- bens DA release (174,245,259), yet other studies reporting no change (29,30,151,194). Studies of VTA DA neuron activity during instrumental training have indicated that presentation of food reinforcement only leads to an increase in DA activity during the initial training period, or when food presentation is novel or unpredictable (139,158,208). If food is regularly presented in an unsignalled manner to animals not making instrumental responses, then food pre- sentation fails to instigate DA neuron activity (139). In well trained animals responding on discrete-trial operant tasks, food presentation fails to elicit a net population response in terms of DA neuronal firing (208). The precise features of instrumental conditioning tasks that lead to increases in DA neuron activity remain unclear. It has been suggested that VTA DA neurons are more responsive to stimuli that signal behaviorally-relevant conditions than they are to the reinforcer itself (139,208). Also, several lines of evidence indicate that accumbens DA activity shows biphasic oscilla- tions during instrumental performance~ with the period of instrumental responding being characterized by an overall increase in responding, but the presentation of the reinforcer being marked by a decrease in DA neuron activity or release ((123,128,166,178), see review in Ref. (191)). 3.4. The role of DA systems in drug self-administration Drug self administration is the area in which DA systems are most frequently linked to the reinforcement process. The General Anhedonia Model is probably the major conceptual scheme that is used'to integrate several diverse findings into a unified approach to the problem of drug abuse. Indeed, the National Institute of Drug Abuse in the United States has spent many millions of dollars on research that is based upon this model. Of course, it is beyond the scope of this paper to review the entire field of drug abuse. Nevertheless, some general features of this literature should be briefly outlined. First of all, it should be noted that there is substantial evidence indicating that accumbens DA is involved in aspects of stimulant self-administration (87,92,100,169,179). There is evidence in favor of the notion that accumbens DA directly mediates the reinforcing effects of non-stimulant drugs, although there also are some findings at odds with that view (49,64,74,85,124,176,200). The question of whether or not accumbens DA is involved in self-administration generally, or stimulant self-administration specifically, is not really the central focus of the present review. Rather, the question being emphasized is whether or not the involvement of accumbens DA in drug self-administration should be used to provide the strongest pillar of support for the General Anhedonia Model. There are several possible explanations of why accumbens DA may be important for drug self administra- tion (e.g. (182,250)). But, does such an involvement neces- sarily provide support for the hypothesis that accumbens DA directly mediates food reinforcement? In fact, there is evidence to the contrary. Roberts et aL (179) showed that accumbens DA depletions that severely affected cocaine self-administration had little effect on lever pressing for food reinforcement. Thus, it is possible that accumbens DA

BEHAVIORAL FUNCTIONS OF NUCLEUS ACCUMBENS DOPAMINE 345 is critically involved in cocaine self-administration, yet the ~00~ General Anhedonia Model is inaccurate because accumbens DA does not mediate food reinforcement. This would 400~- / suggest that the basis of cocaine reinforcement does not ~ lie in the simple assertion that it is based upon the natural ~ 3°°~ ~"~ reinforcement system that evolved to deal with stimuli such ~200~ f . as food. '°°F/ 3.5. Choice and "rate free" measures 0 50 100 150 200 ~50 300 350 400 Another difficulty for the anhedonia hypothesis comes from studies that have focused on response choice measures of the effects of reinforcement. Although reinforcers affect response probability, it also is true that reinforcers control response choice. Several studies have shown that DA antagonis'ts affect response rate or speed in doses that have little effect on response choice. Increasing doses of pimozide increased response latencies in a T-maze light/ dark discrimination task, but did not affect response choice measures (229). More recently, Salamone et al. (192) used a T-maze task in which rats were tested for their ability to discriminate between an arm that contained four 45 mg food pellets and the opposite arm, which contained two food pellets. Although 0.1mg/kg haloperidol substantially slowed run speed, it had no effect on choice of the correct arm. Using a two-lever procedure for studying win-stay hnd lose-shift response strategies, Evenden and Robbins (65) demonstrated that flupenthixol did not affect response choice strategy in a manner similar to extinction, although the drug did slow the rate of response. If the instrumental response involved simply being in proximity to the food dish, a relatively high dose of haloperidol (0.4 mg/kg) dramatically reduced locomotor activity but had no effect on time spent engaged in the reinforced response (185). Several studies showing relatively preserved response choice in neuroleptic-treated rats have used food as the reinforcer (65,185,192,229). Bowers et al. (21) employed a self-stimulation discrimination procedure, and reported that pimozide slowed the rate of responding yet did not alter discrimination choice. Some researchers have suggested that "rate free" mea- sures of behavior, such as reinforcement thresholds, could be used to separate motor and "reward" factors that deter- mine instrumental responding. Essentially, the argument is that conventional response rate measures hopelessly con- found reinforcement and motor processes, and therefore measures should be obtained that more directly represent the rewarding effects of stimuli. Threshold measures and intensity/response functions have been employed in self- stimulation research (61,218,248). Response/reinforcement matching has been used with a variety of reinforcers to obtain a measure of reinforcement value (I 2,95). According to this type of analysis, the relation between reinforcement density and responding on variable interval (VI) schedules is a rectangular hyperbola that basically resembles a dose- response curve. The equation for this hyperbola (see Fig. 1) has been used to fit the relation between reinforcement and responding, and the equation is: B =kR/(R + Re) (1) in which B represents response rate, R represents reinforce- ment density, and k is the constant for maximal responding. Re represents the reinforcement threshold Reinforcements/hour FIG. 1. This figure shows that the relation between reinforcement density and response rate on VI schedules fits a rectangular hyperbola (see Section 6 in text). There are two important parameters (see Eq 1) that are derived from this analysis: the asymptotic response rate (k) and the reinforcement density that generates half-maximal response rate (Re). In this case, k = 500 and Re = 100. (i.e. the reinforcement level that generates 50% of maxi- mum response rate), which is analogous to the ED50 in a pharmacology experiment. Although the "rate free" measures described above have received considerable attention, it is nevertheless true that there are several problems with the interpretation of these kinds of results. The major conceptual problem is that these measures, despite their apparent independence from response rate, are not in fact independent of all aspects of motor function. For example, DA antagonists were reported to increase self-stimulation thresholds (61,111,218), and this effect has been interpreted to mean that these drugs are blocking the reward value of stimulation (248). Some research suggests that DA antagonists decrease reinforce- ment value of food (i.e. increase Re) in matching experi- ments (96,97,171). DA antagonists have been shown to increase sucrose consumption thresholds, or decrease sucrose consumption in a manner similar to reducing sucrose concentration, and these findings have been inter- preted as indicative of a "reward" deficit (103,211). Yet despite these reports, several difficulties remain. It has been shown that the stimulation threshold is not a pure index of reward, and that this measure can be increased by motor factors such as task difficulty (75,81). As noted by Fouriezos et al. (75) "shifts in rate-frequency functions must be interpreted with caution when such shifts are obtained by CNS lesions or drugs". Similar cautions must be maintained when considering results of sucrose consumption thresholds as well. Sensory psychologists discovered long ago that threshold measurements were not "pure" measures of sensation, and i'nstead reflected the interaction between sensory, motor and decision making processes; this is the very reason why signal detection theory (91) has become so popular. Although "rate tree" measures such as stimulation thresholds and reinforcement thresholds from matching experiments do not produce a datum that is itself a response rate, the measure is influenced by motor factors because the threshold intensity still represents a value that is defined by a "response" criterion. Thus, the measure being generated does represent the outcome of a sensorimotor interaction process. This point is particularly vital, because several investigators have suggested that the basal ganglia are very important for the process of sensorimotor integration

346 SALAMONE, COUSINS AND S~qYDEI~ (107,134,145,203,221,224,233,239). Although proponents of the anhedonia hypothesis often state that the deficits produced by low doses of neuroleptics are not sensory or motor (e.g. (211)), this terminology is familiar to those who have studied the sensorimotor functions of basal ganglia. According to Teuber and Proctor (224) "Somehow, the usual distinction of purely motor and purely sensory symp- toms fails us when it comes to an experimental analysis of basal ganglia function and dysfunction. It is almost as if we needed two categories to describe some of the symptoms which are neither sensory nor motor but reflect peculiarities of sensorimotor interaction". Behavioral deficits in DA depleted and haIoperidol-treated rats, as well as those shown by human parkinsonian patients, can be partially ameliorated by providing additional sensory stimulation (140,170,209). Rats that are akinetic due to striatal DA 'depletions combined with injections of haloperidol can nevertheless respond to intense sensory stimuli (115). The increase in stimulation or reinforcement .threshold produced by DA antagonists simply means that a higher level of stimulation is necessary for producing a response, which is indicative of a drug-induced decrease in responsiveness to stimulation (239). As noted by White (239), early studies of the effect of DA depletion on sucrose consumption thresh- olds could be interpreted as reflecting sensorimotor deficits induced by DA depletion. According to Muscat and Willner (162) "The 'dopamine hypothesis of (sweet) reward' may thus be a misrepresentation of data derived under limited conditions of reinforcement. The blunting of reward by DA antagonists may be a special case of a more general attenuation of the influence of sensory stimuli over behavior". Therefore, the reduction in behavioral reactivity to conditioned stimuli produced by DA antagonists or depletions can easily be interpreted within the general framework of the sensorimotor functions of DA systems (5,18,28,44,188-190). There are additional problems with drug-induced changes in curve fitting parameters such as Re being interpreted as equivalent to drug-induced reductions in the reward value of a stimulus (186,189). The attn'bution of reward-related effects to actions on Re is based upon the idea that k (maximal response rate) is the only parameter that is influenced by motor factors and Re is only influenced by motivational factors. In fact, there is some dispute about these points in the matching literature (e.g. (98,241)). Also, it is questionable whether increases in Re should be viewed as an irreducible index of decreases in the reinforcement value of a stimulus, such as food. As noted above, Re is a parameter that is equivalent to the ED50 in a drug experi- ment. Yet, ED50s are clearly influenced by a variety of factors, including affinity for a receptor, duration of action and penetration into the target tissue. It is possible that a number of factors, including some related to aspects of motor function, could influence the apparent Re. Baum (12) reported that responding on VI schedules in response/ reinforcement matching experiments could by influenced by response-related factors such as effort or response preferences (i.e. response bias). Thus, another way of describ- ing the effects of DA antagonists in matching experiments would be to state that interference with DA may not be directly affecting reinforcement value, but instead may be altering the bias between responses with different requirements (186). Williams (241) gives equations for hyperbolae that include a measure of bias (b). In these equations B = k bRl(bR + Re) or B = k RI(R + Re~b) it can be seen that the apparent Re value from Eq. 1 could be represented by a composite parameter (Re/b in Eq. 3) that is influenced by bias. In fact, previous matching experiments studying the effects of neuroleptics have basically assumed that bias is either unimportant, or thatit remains unchanged by drug treatment. One can fit Eqs 2 or 3 to neuroleptic data, and if one assumes that Re does not change, then the effects of DA antagonism can be fitted to a drug-induced decrease in b (i.e. a bias away from lever pressing and towards other, less vigorous responses). A related way of interpreting effects upon Re is to point out that increases in Re do not necessarily mean decreases in the reinforcement value of a food stimulus. Instead, increases in Re reflect a decrease in the overall reinforcement value of lever pressing for and consuming food. Thus, neuroleptics could be producing sen- sorimotor effects that make the motor activity of lever press- ing less reinforcing. According to the regulatory or motivational view or reinforcement (see below), this would mean that the motor activity of lever pressing would be less preferred, or that the preferred level of lever pressing activity would be decreased. Clearly, this is an interpretation that is not really consistent with the anhedonia model or the absolute reward/motor distinction. " 4. WHAT IS "REINFORCEMENT"? (2,3) I ! I ! ! 4.1. Behavioral approaches to the concept of "reinforcement" One of the greatest problems with the hypothesis that DA in nucleus accumbens directly mediates reinforcement is the fact that the meaning of the term "reinforcement" is not often discussed in detail in the neuropharmacology litera- ture. As implied by the very use of the term "hedonia", reinforcement is often treated as being synonymous with "pleasure". Indeed, a close examination of the literature dealing with the General Anhedonia Model indicates that this connection between reinforcement and pleasure was no accidental and unfortunate slip of the pen. As discussed above, it clearly was intended that the effects of neuroleptics on instrumental behavior be ascribed to actions on emotional processes and not motor functions (e.g. (248,253,254). Of course, anyone familiar with the behavioral literature on instrumental conditioning would recognize that subjective pleasure is not usually emphasized as the defining characteristic of the process of reinforce- ment. Clearly not compatible with Skinnerian behaviorism, such hedonic views of reinforcement also have a long tradition of being rejected by learning theorists (99,155). The most common description of reinforcement is that developed by Skinner, which is known as the Empirical Law of Effect. According to this view, which is a modifica- tion of Thorndike's original Law of Effect, the defining characteristic of reinforcement is rooted in the relations between response probabilities and the presentation of stimuli. Thus, if a response is followed by presentation of a stimulus, and the response thereafter shows an increase in relative probability, then the process of positive reinforce- merit is said to have occurred. The stimulus that was presented to produce this effect is known as a reintbrcer. i ! I I

d ,). ~EHAVIORAL FUNCTIONS OF NUCLEUS ACCUMBENS DOPAMINE 347 Throughout his long and productive career, Skinner focused on specifying the relations between reinforcement and response output, and he rarely considered the question of which specific characteristics determine the properties of reinforcing stimuli. Of course, if one maintains that a parti- cular brain structure mediates reinforcement, then the. pro- blem of defining the basic characteristics of reinforcing stimuli would have to be one of the most critical questions to be considered. Several behavioral researchers have addressed this question, and despite the differences between each of their particular views, it is important to emphasize the common elements that are apparent across several different investigators. A number of researchers have emphasized that reinforcers are stimuli towards which the behavior of organisms is directed, i.e. organisms will approach, consume, interact with, or increase the proximity or availability of reinforcing stimuli. Thorndike (225) defined a "satisfier" as a stimulus that the "the animal does nothing to avoid, often doing such things as to attain and preserve it". According to Premack (173), reinforcing activities occur with a relatively high probability, or are relatively preferred. Bindra (17) and Glickman and Schiff (86) emphasized that reinforcers elicit approach and con- summatory responses. According to response deprivation theory (7,227) reinforcing activities can be described as being relatively deprived, and the process of reinforcement involves the restoration of the response equilibrium that was disturbed by deprivation. Consistent with the role of rein- forcers in regulatory or homeostatic processes, some researchers have emphasized the role of reinforcement in providing feedback that guides motor output (226). In a recent review (189) it was noted that this "motivational" or "regulatory" view of reinforcement actually is quite con- sistent with Skinner's Empirical Law of Effect. Skinner defined reinforcement in terms of the effects of a stimulus on response probability, while the motivational perspective is centered on the behavior of the organism and how it modifies its environment. If there is a contingent relation between an instrumental response and a reinforcer, then it must be true that the organism is increasing the probability of reinforcer presentation by engaging in the instrumental response. As observed previously "a motivational corollary of the Empirical Law of Effect is that a reinforcer is a stimulus that is increased in probability by the organism" (189). This fundamental property of reinforcers to elicit approach responses has been referred to as the uncondi- tioned rewarding or reinforcing property of a stimulus (217). Behavioral investigators also emphasize other character- istics of reinforcing stimuli. As noted by Mackintosh (141,142), organisms do not simply approach reinforcing stimuli. Animals are capable of learning very specific associations between an instrumental response and a stimu- lus. and learn to repeat a form of the previously reinforced response in the future. According to Timberlake and Allison (226,227), even if response deprivation describes the regu- latory basis of reinforcement, animals would still need to learn "what leads to what". This would indicate that response/reinforcer associative processes also are important for the process of reinforcement (37,142,177). Moreover, instrumental responses are instigated not simply in the presence of the reinforcer, but they typically are elicited by conditioned stimuli that occur when the organism is temporally and spatially distant from the reinforcer. These conditioned stimuli form another aspect of the associative structure of instrumental behavior (177). Another important aspect of reinforcers is that they can have behaviorally activating properties, which have several manifestations. First, the periodic presentation of a reinforcer such as food to a food deprived animal can result in high levels of motor activity (119,120). This phenomenon can result in the induction of a variety of different "adjunctive" behaviors, and also serve to support operant response rates (121). In addition, organisms can perform very high levels of motgr output in order to gain access to reinforcers. Rats will run in alleys or mazes, vigorously press levers or forage over wide areas in order to gain access to food or other significant stimuli. To summarize, reinforcers have a number of dif- ferent characteristics. These include: (1) positive reinforcers are stimuli towards which behavior is directed, and organ- isms behave in such a way as to increase the probability of (i.e. self-administer) such stimuli; (2) reinforcer presen- tation is one of the stimulus events that is embedded in a complex associative structure linking stimuli and responses; (3) reinforcers provide feedback control over motor output; and (4) reinforcers can have behaviorally activating effects. It is probably true that instrumental behavior is not an elemental conditioning process, but. rather is a complex phenomenon that emerges from the interaction of the factors mentioned above, as well as other conditions (e.g. biological constraints). 4.2. DA and reinforcement: a reexamination In light of this overview of the characteristics of reinfor- cing stimuli, it is important to reconsider the data linking DA to food reinforcement. First of all, it should be empha- sized that DA, particularly in nucleus accumbens, has been implicated in the process of behavioral activation (24, 118, 127, 151,159, 180, 185-191,242,243,250). Thus, accumbens DA clearly is involved in aspects of the reinfor- cement process. Yet, difficulties emerge when one considers the specific nature of such an involvement. Does dopami- nergic involvement in behavioral activation mean that DA mediates hedonia? Indeed, the "reward" vs "motor" dis- tinction begins to take on the appearance of an artificial dichotomy when one considers that several of the behavioral characteristics of reinforcing stimuli are fundamentally connected to aspects of sensorimotor function. Behavioral activation clearly represents an area of overlap between motivational and motor function, and several other aspects of reinforcement processes also fall into a gray area between learning and motor processes. Instrumental conditioning is fundamentally a form of motor learning. The ability of reinforcers to provide feedbaclt control over responding, and the ability of responses and reinforcers to become associated, reflect higher order sensorimotor integrative functions that may require the involvement of basal ganglia DA (28,170). Thus, it is important to distinguish between the literature that implicates DA in aspects of the reinforce- ment process from the notion that accumbens DA is involved in hedonic aspects of reinforcement that have nothing whatsoever to do with any aspect of motor function. As discussed above, appetitive motivation is considered to be a fundamental aspect of the reinforcement process. Thus, it is important to discuss the effects of DA antagonism and accumbens DA depletions on food acquisition and

348 consumption. Consummatory behaviors have sometimes been used as a measure of "reward" or "reinforcement" (103,162,251,252). Proponents of the General Anhedonia Model have emphasized the effects of neuroleptic drugs on the maintenance of food consumption (248), and on the unconditioned responses elicited by food (248). Wise (248,249) has relied very heavily upon Bindra's ideas of incentive motivation, and clearly has supported the notion that appetitive motivation is an important aspect of reinfor- cement. But a close examination of the literature in this area causes some difficulties for the anhedonia model. Sucrose consumption is impaired by DA antagonists (204-206,211) and it has been argued that this effect is not attributable to motor problems because the local frequency of licking is not altered by neuroleptics. Yet, it appears as though the local rate of licking is a poor indicator of the motor effects of DA antagonists, because the lick rate is unaffected even by cataleptic doses of haloperidol (78). It is likely that the lick rate is set by brainstem pattern generator mechanisms (78,240), and that basal ganglia DA may have little effect on that particular parameter. Other motor parameters related to licking are affected by DA antagonists, including lick efficiency (103,204,206), lick duration (78,80,88) and lick ".force (78,80). Jones and Mogenson (112) reported that injections of low doses of spiroperidol directly into nucleus accumbens impaired lap volume and tongue extension in a water licking procedure. Also, the attribution of sucrose consumption deficits to reward impairments seems uncer- tain in view of the studies showing that whole forebrain DA depletions or acute neuroleptic administration did not alter appetitive taste reactivity to sucrose (16,231). Higher doses of DA antagonists suppress chow feeding, and evidence indicates that this effect is largely attributable to motor problems that reduce the rate of feeding and impair food handling (4 I, 199). Several DA antagonists have been shown to decrease feeding rate (32,41,199), and pre-feeding to reduce food motivation predominantly affects feeding dura- tion, with little or no effect on feeding rate (40,199). In a direct comparison, the effects of pre-feeding on chow intake differed substantially from the effects of haloperidol (199). It has been demonstrated that neuroleptic drugs can sub- stantially reduce lever pressing for food at lower doses that do not impair simple food approach responses or food con- sumption (19,73,183,185). These results are typically inter- preted to mean that a fundamental aspect of food reinforcement (i.e. the tendency to approach and consume food) is intact in rats treated with moderate doses of DA antagonists. Perhaps the first report of this phenomenon was by Rolls et al. (183), who interpreted their findings to mean that DA antagonists interfered with "complex motor responses". Another early report was by Fibiger et al. (73), who stated that "the decreased bar pressing for food was not the result of anorexia or reduced motivation for food", and later that DA antagonists were not "interfering with reward". Ljungberg, in several papers, has shown that DA antagonists impair water-reinforced lever pressing at doses that do not impair water intake (136-138). In the initial report (136), it was stated that "'the attenuation of operant responding after low doses of neuroleptics cannot be explained by an effect on the ability of animals to regulate their body water (i.e., on ~motivation' or on 'reinforcing properties' of the water)." Blackbum et al. (19) also obtained related findings, in that they observed that SALAMONE, COUSINS AND SI~YDER" food-related anticipatory reactions were more greatly impaired than food consumption by low doses of pimozide. These authors concluded that their results could not be attributed to a "reward" deficit. In view of the fact that low doses of DA antagonists that suppress lever pressing for food do not reduce food consumption, it seems difficult to argue that the unconditioned reinforcing properties of food must be suppressed in order for DA antagonists to impair lever pressing for food. The effects of DA antagonism on basic processes of stimulus-stimulus associations remain uncertain, and some aspects of associative processes may be impaired while others remain intact (4,14,235). Studies of classical conditioning indicate that DA antagonists appear to blunt the conditioned and unconditioned excitatory or arousal properties of stimuli (93). Yet in the context of this discus- sion, it should be stressed that there is quite a diffe~'ence between the "excitatory properties of stimuli" and "hedo- nia"; DA antagonists could be altering arousal functions that are related to aspects of associative processes yet still not directly affect primary reinforcement. In discussing the effects of flupentixol on the development of place prefer- ence, Agmo et al. (4) stated that although the DA antagonist may have affected associative processes, the doses given did not affect sucrose consumption and therefore "the reward value of food or water was not reduced". Another line of investigation has involved attempts to assess the behavioral reactivity of neuroleptic-treated rats to food presentation. The locomotor activity induced by scheduled food presentation is substantially reduced by haloperidol (i 87). Yet other aspects of behavioral reactivity to food are relatively intact in neuroleptic-treated rats. Berridge et al. (16) demonstrated that extensive forebrain DA depletions that involved both striatum and nucleus accumbens did not affect appetitive taste reactivity to sucrose solutions. In a more recent study, it was shown that acute neuroleptic administration did not alter appetitive taste reactivity to sucrose (231). Kirkpatdck and Fowler (122) studied the relation between emitted lever pressing force and the concentration of sucrose reinforcement. Typi- cally, untreated rats emit greater forces in response to higher sucrose concentrations, and administration of pimozide failed to affect this pattern (122). An operant psychophysi- cal procedure was employed by Martin-Iverson et aL (147) to study the effects of haloperidol on perceived reward quantity, and this DA antagonist failed to alter responding in a manner consistent with a reduction in perceived hedonic value. In summary, several lines of evidence indicate that low doses of DA antagonists that can suppress lever press- ing for food do not impair the primary reinforcing properties of food. 4.3. Behavioral effects of accumbens DA depletions Much of the work related to the anhedonia hypothesis has focused upon the effects of systemically administered neu- roleptics. Because nucleus accumbens DA has specifically been implicated in food reinforcement, it is important to examine in detail the literature linking accumbens DA to food-related behaviors. Given the hypothesized importance of'accumbens DA for food reinforcement, it is relevant to consider that accumbens DA depletions, intra-accumbens injections of haloperidol and large ibotenic acid lesions of

~ B 'EHAVIORAL FUNCTIONS OF NUCLEUS ACCUMBENS DOPAMINE 349 accumbens actually have little effect on lab chow consump- tion (11,125,197,233,236). Nucleus accumbens DA deple- tions did not affect total food intake, total time spent feeding, feeding rate or forepaw usage during feeding (197). Recently, it was shown that accumbens DA deple- tions did not alter discrimination of reinforcement magni- tude, nor did they affect response selection in a T-maze task based upon reinforcement magnitude (192). Although the focus of the present review is upon food motivation., it also is useful to examine the role of accum- bens DA in sexual motivation. Interference with accumbens DA does not affect preference for female sex partners in male rats as assessed in a maze choice task (104). Yet interference with accumbens DA does slow down the rate at which male rats run through a maze to approach female rats (104). Depletions of accumbens DA have little effect on male sexual behavior, but one of the effects is a slowing of the initiation of copulation (67). Although periods of lordo~ sis may be coincident with increases in accumbens DA release (156), DA depletions do not suppress lordosis. Thus, as shown with food-related tasks, interference with accum- bens DA does not substantially alter the consummatory behaviors associated with primary sexual motivation, and leaves important aspects of sexual reinforcement intact. A few experiments have examined the effects of accum- bens DA depletions on instrumental lever pressing. Although substantial increases in accumbens DA release accompany lever pressing on a CRF schedule, the effects of accumbens DA depletions on total number of CRF responses emitted are actually very mild and transient (149,195). Despite the fact that the total number of lever presses on a CRF schedule is only marginally affected by substantial DA depletions, some parameters of responding are affected considerably. Accumbens DA depletions pro- duce an initial reduction of response rates during CRF sessions, which results in a slow but steady rate of respond- ing throughout the session that does not resemble extinction (149,195). Depletions of accumbens DA also produce a slowing of the local rate of responding on both CRF and fixed ratio (FR) schedules, as measured by the analysis of interresponse times (195,196). The slowing of the local rate of FR 5 responding produced by accumbens DA depletions can last for several weeks after surgery, even though the total number of responses tends to recover more quickly (196). Although accumbens DA depletions have little effect on lab chow intake, they do substantially reduce the motor activity induced by periodic food presentation (151,197). Amphetamine-induced enhancement of responding sup- ported by conditioned reinforcers is reduced by accumbens DA depletions (223), and several studies have linked accumbens DA to conditioned incentive processes (25,26,68,69,117,182). Yet local injections of D1 or D2 agonists into the nucleus accumbens that affect conditioned reinforcement have no effect on intake of the water reinforcer, indicating that "alterations in primary moti- vation do not underlie the changes in response to condi- tioned reinforcement" (255). Future research should focus on the specific "core" and "shell" subregions of accum- bens to determine the degree of regional heterogeneity of function (53,143,157,262,263). Yet some conclusions can be drawn based upon the information available so far. DA depletions in the core region appear to have little effect on the unconditioned reinforcing properties of food. Nevertheless, DA depletions of accumbens core do affect processes related to reinforcement; DA depletions do pro- duce a slight slowing of instrumental responding, reduce behavioral activation and blunt behavioral reactivity to conditioned stimuli. 5. COST/BENEFIT ANALYSIS: INVOLVEMENT OF DA IN THE RELATIVE ALLOCATION OF RESPONSES IN RELATION TO CONSTRAINTS 5. I. Behavioral economics of reinforcement processes As discussed above, behavioral researchers have focused upon the characteristics of stimuli that function as reinforcers. Although researchers usually discuss individual reinforcing stimuli in isolation, it should be recognized that the environment is rarely so simple; organisms typically make choices between a variety of different reinforcers. Research into response/reinforcement matching has emphasized that reinforcement is a relative, and not an absolute, process. Thus, an important aspect of reinforce- ment is that animals select activities based upon their reinforcement value relative to other currently available sti- muli. Yet despite the importance of reinforcement value as a determinant of stimulus selection, it should also be recognized that there are response-related factors as well. The vast array of reinforcing stimuli are rarely if ever present in an uncon- strained environment. Organisms must perform instrumental responses to gain access to reinforcers, and these responses typically involve work. Several behavioral investigators have emphasized the importance of response "costs" or "con- stralnts" for determining instrumental response selection (34,83,106,114,153,175,186,189, 214,215). In view of the role of both reinforcement value and response costs in determining instrumental responding, it is important to consider that instrumental behavior basically involves ongoing choices between reinforcers of different values contingent upon responses with different costs. Such cost/benefit or "economic" approaches have become more common in studies of instrumental behavior (6,7,35, 105,132). Moreover, it should be recognized that cost/ben- efit analyses are generally important in a number of different fields, including various aspects of psychology (267), bio- logical studies of optimal foraging (129,130,216) and, of course, economics. Considering that DA systems have been implicated in behavioral activation, it is important to ponder the role of DA in the "cost" side of the cost/benefit interaction (163). It is possible.that accumbens DA is not involved in directly mediating reinforcement value per se, but instead is involved in enabling organisms to overcome response costs or constraints. It has been suggested that accumbens DA is involved in the process of allocating responses in relation to various reinforcing stimuli (102,186,188-190). As reviewed below, research with var- ious cost/benefit tasks has indicated that interference with accumbens DA alters the allocation of instrumental responses away from more vigorous or effortful responses and towards the selection of less effortful responses. 5.2. hzvolvetnent of DA in the relative allocation ~ respotlses In order to study food-related responses with different costs, a behavioral choice procedure was developed that

350 SALAMONE, COUSINS AND SN~/DER increases in chow consumption have been shown after • .~---....~ ~ Chow acute injections of the D2 antagonist haloperidol, the D I l,~[- ~ × Pellets ~ antagonist SCH 23390, and the non-selective DA antagonist ~til ~ flupentix°l (47)" Thus' DA antag°nists with a wide variety !i of characteristics all have been shown to decrease lever pressing and increase chow consumption. Several drugs have been investigated that do not produce the same pattern ~ of effects on the FR5/chow feeding task. The stimulant and appetite suppressant, amphetamine, suppressed both lever 0~- ~ pressing and chow intake (47). The muscarinic agonist FR1 FR5 FR10 FR20 FIG. 2. Demand curve showing the effect of ratio on food obtained through lever pressing (open squares, "pellets") and lab chow consumption (x, "chow") for a group of rats (n = 24) tested on the concurrent lever press- ing/feeding task. These rats had been trained initially using FRI, 5, 10 and 20 schedules without any chow present, after which they were tested on FR1, 5, 10 and 20 schedules with lab chow concurrently available. Each rat spent I week on each schedule component, and schedule order was counter- balanced across rats. The data shown represent the mean food intake for an' entire week under each ratio schedule, and the overall effects of schedule on Bioserve pellet and lab chow consumption were statistically significant (see Snyder, B., unpublished Masters Thesis, University of Connecticut, USA). Data for lever pressing arc shown as Bioserve pellet intake (no. of reinforcers × 45 mg) so that lever pressing and chow data could be shown on the same scale. has allowed for the concurrent assessment of the effects of DA antagonists or DA-depleting brain lesions on lever pressing and food consumption (198). In this instrumental lever pressing/feeding procedure, a preferred food (Bioserve pellets) is available by pressing a lever on a fixed ratio (FR) schedule, while a less preferred food (lab chow) is also available concurrently in the operant chamber. Rats show a strong preference for Bioserve pellets over lab chow, and haloperidol did not affect this preference in free feeding preference tests (198). In versions of the FR/feeding choice task that involve low ratio values (i.e. FR1 or FR5), untreated or control rats typically obtain virtually all of their food by lever pressing for the preferred food, and eat little of the available lab chow (44,45,196). Increasing work requirement by using higher ratio values such as FR10 and FR20 leads to a shift in behavior away from lever pressing to obtain the preferred food and towards the acquisition and consumption of lab chow. Figure 2 is a "demand curve" showing the relation between ratio value and the intak~ of food obtained both from lever pressing and chow con- sumption. In this figure it can be seen that, in the range from FR1-FR20, as the ratio value increases food obtained from lever pressing decreases and chow consumption increases. "Thus, the concurrent lever pressing/feeding proce- dure has some of the characteristics of a cost/benefit task, in the sense that this procedure has been shown to be sensitive to increasing work requirements. Several studies have been performed with the instrumental lever press/feeding task, most of which have employed the FR5 schedule as the requirement for obtaining the preferred food. Using FR1 and FR5 versions of the lever press/feeding task, systemic administration of haloperidol was shown to reduce lever pressing but significantly increase chow con- sumption (198). This effect did not mimic the characteristics of reduced food motivation, as it was shown that pre-feeding acted to reduce both lever pressing for food and chow consumption (198). Decreases in lever pressing and pilocarpine failed to increase chow intake in doses that did suppress lever pressing (47). Sodium pentobarbital increased chow intake and decreased lever pressing at only one dose of all those investigated (47), which stands in marked contrast to the DA antagonists that suppress lever pressing and increase chow intake over a 2-3-fold dose • range. Thus, these results show that all drugs that suppress lever pressing do not act over a wide dose range to increase the intake of chow that is concurrently available. The fact that several DA antagonists decrease FR5 lever pressing and increase chow intake indicates that these drugs can suppress lever pressing in doses that do not produce profound deficits in food motivation or sedation that would also act to suppress food intake. The haloperidol-induced shift in behavior from lever pressing to chow intake also manifests itself as a high inverse correlation between lever pressing and chow intake across individual animals after acute administration (193). Thus, animals that show greater suppressions of lever pressing after haloperidol administration also show greater increases in chow consumption• The impairments in lever pressing produced by haloperidol, SCH 23390 and flu- penthixol are occurring at low/moderate doses that leave the rats directed toward the acquisition and consumption of food, which indicates that the lever pressing deficit is not due to a general loss of food motivation. The relative preservation of appetitive motivation in neuroleptic-treated rats is important, considering that many investigators have suggested that appetitive motivation provides a fundamental basis for the process of reinforcement (see discussion above). These results have been interpreted to mean that moderate doses of DA antagonists decreased lever pressing but did not alter the unconditionally rewarding character- istics of food consumption (44,45,47,198). 5.3. The role of accumbens DA #~ response allocation In order to study the brain mechanisms involved in the performance of the concurrent FR5/feeding task, the effects of DA depletions in different brain regions have been investigated. Considerable research has identified the nucleus accumbens as the critical brain locus for producing the decrease in lever pressing coupled with an increase in feeding (44,45,198)• Injections of haloperidol directly into the nucleus accumbens decreased lever pressing and increased chow consumption (198). The neurotoxicant 6- hydroxydopamine (6-OHDA) has been injected directly into several DA terminal regions, including the nucleus accum- bens and neostriatal sites, in order to determine the effects of local DA depletion on pertbrmance of the FR5/feeding task. Depletions of DA in accumbens lead to a dramatic shift in behavior, in which there is a significant decrease in FR5 lever pressing but a significant increase in consumption of

B~HAVIORAL FUNCTIONS OF NUCLEUS ACCUMBENS DOPAMINE 351 d lab chow (44,45,198). DA depletions in the medial striatum did not significantly affect lever pressing or chow consump- tion, and the nucleus accumbens is the only site at which DA depletions mimic the effects of administration of low doses of systemic neuroleptics. Thus, depletions of nucleus accumbens DA do not produce a general reduction in food motivation, although accumbens DA depletions do decrease instrumental lever pressing for food when alternative food sources are available (44). Depletions of DA in the ventrolateral neostdatum (VLS) reduced both lever pressing for food and chow consumption (44). Based only upon this result, it may appear superficially that the VLS is the long sought "reward center" for food. Yet such a statement is without any basis in fact; there is no evidence whatsoever that VLS has any "reward" functions related to food, and there is an enormous body of evidence demonstrating that the VLS is involved in skilled motor usage of the forepaw and head regions. In an initial study, (110), it was observed that DA depletions in the VLS produced profound feeding deficits, and also produced tremulous oral movements. In a subsequent study, 6- OHDA was injected into the nucleus accumbens, antero- ventromedial striatum, and VLS. Detailed behavioral observations demonstrated that the 6ritical region in which DA depletions disrupt feeding is the VLS; rats with VLS DA depletions have pronounced deficits in food intake, feeding rate and forepaw usage during feeding (197). Thus, these results indicate that a classic result of striatal DA depletion (i.e. feeding deficits (60,146,219,232) can be localized to a particular striatal subregion, i.e. the VLS). Rats With VLS DA depletions are directed towards food consumption; most of these rats will vigorously consume wet mash, will attempt to eat large dry pellets, and will attempt to grasp small food pellets (184,197). Yet, rats with VLS DA depletions have severe motor/sensorimotor impairments that interfere with their ability to reach, grasp, handle and chew pellets (197). This conclusion is consistent with previous work demonstrating that severe impairments in reaching and grasping with the contralateral forepaw are produced by DA depletions in lateral striatum (66,237) or, more specifically, the VLS (184). Also, these results are consistent with anatomical data showing that the lateral neostriatum receives massive projections from sensorimotor neocortex (36,154). Additional experiments have been conducted to characterize the effects of local DA depletions on the microstructure of operant responding. Computerized behavioral analyses were conducted to obtain various measures of instrumental responding, includ- ing the interresponse time (IRT) for each operant response. Each IRT represents the reciprocal of the local response rate. As noted above, accumbens DA depletions produced a slight decrease in FR5 lever pressing and a modest slowing of the IRT distribution. In contrast, VLS DA depletions produced profound motor deficits that resulted in substantial and persistent reductions in lever pressing and a dramatic slowing of the IRT distribution (196). Subsequent work has shown that VLS DA depletions alter both the initiation and duration of lever pressing responses (46). Thus, VLS DA depletions clearly produce profound deficits in skilled motor usage, which affect the ability of rats to eat large food pellets or press levers to obtain food. In contrast, the effects of accumbens DA depletions are much more subtle. Accum- bens DA depletions do not impair chow intake, and only slightly affect lever pressing on CRF or FR5 schedules. Nevertheless, when chow is available concurrently along with lever pressing on FR5 schedules, accumbens DA depletions alter the path that the animal uses to obtain food, such that lever pressing is decreased and chow con- sumption is increased. One of the major goals of this research was to determine whether accumbens DA depletions caused a decrease in lever pressing and an increase in chow consumption because of motor deficits that set an absolute limit on the number of lever presses that could be emitted. In a recent study (45), rats were tested on days 1, 3, and 5 of a 5-day test week using the concurrent FR5/feeding procedure. On days 2 and 4 of each week lab chow was not concurrently available, and rats could only lever press on the FR5 schedule for pellets to obtain food. DA depletions produced by intra-accumbens injections of 6-OHDA produced dramatic decreases in lever pressing and increases in chow consumption on days when lab chow was available. Yet, lever pressing was not sig- nificantly reduced in DA-depleted rats on days when chow was not available, although there was a significant correla- tion between lever pressing and accumbens DA levels. These results again suggest that accumbens DA depletions do not produce a general deficit in food motivation, ankl also indicate that accumbens DA does not appear J~o be critical for the execution of individual motor acts involved in lever pressing. Therefore, the shift from lever pressing to chow consumption following accumbens DA depletions is not due to a motor deficit that sets an absolute ceiling on the number of responses that the rats could emit. Rather, these results are consistent with the notion that accumbens DA regulates the higher-order processes that are involved in response allocation relative to a variety of motivational stimuli. 5.4. A T-maze versiori of the cost~benefit procedures: involvetnent of accumbens DA in crossing energy barriers Although many studies have been performed using the instrumental FR5/chow feeding procedure, several impor- tant questions remain. Possibly, the shift from lever pressing to chow consumption that was produced by accumbens DA depletions could have occurred because these DA deple- tions selectively affect only "instrumental" behaviors such as lever pressing, but do not'affect "consummatory" behavior. In considering this question, it should of course be emphasized that food consumption also relies upon simple instrumental responses such as approaching and remaining in proximity to food (198,252). Yet despite such considera- tions, it is important to investigate cost/benefit procedures under conditions in which explicit instrumental responses are being selected. To address these concerns, a novel T- maze version of the cost/benefit paradigm was developed (192). For this task, one arm of the T-maze contains an alleyway that leads to two food pellets, whereas the other arm contains a 44 cm wire barrier that must be climbed in order to reach four food pellets (see Fig. 3). Well-trained rats continue to cross the barrier to receive four pellets. A low dose of haloperidol caused animals to shift away from the arm with the barrier, and towards the no-barrier arm that had a lower magnitude of reinforcement. However, this dose of haloperidol did not cause a shift away from the arm with four pellets if there was no barrier present. Accumbens DA depletions also decreased selection of the arm that contained

352 Top view Barrier ~ FIG. 3. Top view of the T-maze apparatus descibed in the text is shown. The start arm of the maze was a box 29 × 21 × 21 era. The test arm ar~.a was a box 99 x 32 X 59 cm, which was constructed of wooden walls and a wire mesh floor. The connection between the start arm and the test arms was a sliding aluminium door. The barrier (shown here on the left) was 44 cm high and was made from a metal grating with parallel bars 2.5 cm apart. the barrier, and increased selection of the no-barrier arm that contained only two pellets. Yet accumbens DA depletions did not alter selection based on reinforcement magnitude when there was no barrier present in either ann. These results closely resemble those obtained from the operant procedure, and indicate that the shift away from the arm with the barrier is not due to a loss of preference for the higher reward magnitude (192). Halopeddol-treated and DA depleted rats remain directed towards food acquisition and consumption, yet these animals altered their behavior to select the less effortful path to obtain food. A subsequent study was also performed using the T-maze barrier task (43). Under one test condition, one ann of the maze contained a high reinforcement density (four × 45 mg Bioserve pellets) and the other arm contained a low rein- forcement density (two × 45 mg pellets). The barrier was placed in the arm that contained the high density of food reinforcement. In the second test condition, a separate group of rats was trained in the same T-maze, in which there were four food pellets in the arm that was obstructed by the barrier, yet there were no food pellets in the unobstructed arm. After training rats received intra-accumbens of injec- tions 6-OHDA or ascorbate vehicle. Accumbens DA deple- tions substantially decreased the number of selections of the obstructed arm with the high reinforcement density when the unobstructed arm also contained two food pellets. DA- depleted rats in this condition showed increased selection of the no-barrier arm as well as decreased entry into the arm that contained the barrier. These effects persisted through- out the three weeks of post surgical testing. Nevertheless, when the unobstructed arm contained no food pellets, and the only way to obtain food was to climb the barrier, rats with accumbens DA depletions showed only a modest effect on choice of the obstructed arm, which recovered by the second week of testing. DA-depleted rats that were tested with food in the unobstructed arm showed significantly fewer barrier crossings than DA-depleted rats that were tested with no food in the unobstructed arm. Thus, these findings were not consistent with the notion that accumbens DA depletion rendered the animals unable to climb the SALAMONE, COUSINS AND SI~YDER" barrier, or set an absolute ceiling on the number of barrier crossings the animals could perform. Instead, these results indicated that accumbens DA depletions affected the relative allocation of barrier climbing responses if alterna- tive food sources were available. Although rats with accumbens DA depletions crossed the barrier if this was the only source of food, these rats did have significant latency deficits (43). Interestingly, latency to leave the start box was significantly longer in this group, despite the fact this particular measure would not be affected directly by the time it takes to climb the barrier. Thus, DA depleted rats that eventually crossed the barrier took longer to inititate their responses. It is possible that this increased latency to leave the start box reflects a deficit in the planning stage of a complex movement such as barrier climbing, although the DA-depleted rats do eventually leave the start chamber and climb the barrier. 6. CONCLUSIONS The behavioral functions of nucleus accumbens DA remain complex and enigmatic. Certainly, it is unwise sim- ply to describe the functions of accumbeps DA with the unqualified use of such terms as "motor", "motivation", "reinforcement" or "reward". These terms are extremely blunt instruments, and they are inadequate for the task of precisely defining the functions of this structure. As noted above, there is considerable evidence that would cause one to question the notion that accumbens DA directly mediates the primary "rewarding" effects of food stimuli. Thus, it may be most useful to try and resist the current "zeitgeist", which would have one simply state that DA in nucleus accumbens directly mediates "reward". The processes of reir~forcement, motor control and moti- vation are not irreducible in nature; each term refers to several different components. Moreover, there is consider- able overlap between these processes (131,189). The gen- eration of,responses is affected by reinforcement, but it is also an aspect of motor control and sensorimotor function. The ability to form associations between stimuli and responses is an aspect of reinforcement, an aspect of learn- ing and an aspect of sensorimotor integration. The beha- vioral activation induced by motivational stimuli is an aspect of motivation, an aspect of reinforcement and an aspect of motor control and sensorimotor responsiveness. The work of Zeigler (264-266) has shown that lesions of sensory trigeminal circuits disrupt both sensorimotor and motivational aspects of feeding. One of the difficulties in identifying the behavioral functions of accumbens DA is that its functions lie in the areas of overlap between motivational and motor processes. Thus, one way of "describing the functions of accumbens DA would be to state that it is involved in higher order motor and sensor- imotor processes that are important for activational aspects of motivation, response allocation, and responsiveness to conditioned stimuli. Unfortunately, this may not roll off the tongue quite as fluently as the word "reward". Never- theless, the statement given above may be more useful and informative than any single word yet invented in the English language or technical lexicon. Essentially, it is being argued that the absolute distinction between "motor" and "reward" functions of DA, an idea which is so critical for the General Anhedonia Model, has I I I I I°

BEHAVIORAL FUNCTIONS OF NUCLEUS ACCUMBENS DOPAMINE 353 in 10 tO has reached the end of its useful life and should be discarded. ~For the overlap between motor and motivational years, pro- cesses has been emphasized by behavioral researchers (57,261). Cofer and Appley (33) suggested that conditioned ~stimuli activated an "anticipation-invigoration mechan- ism" that induced motor activities and increased the vigor of instrumental responding. These important behavioral functions, which for so long were described in the absence ~of any understanding of brain mechanisms, seem highly suitable as descriptions of the behavioral functions of accumbens DA. It has been suggested that DA systems are important for overall energy regulation (45,186,222). ~lIn contrast to the use of the term "anhedonia", the term "anergia" has been used to describe the effects of DA antagonism and accumbens DA depletion (192). [[Various phrases have been used to describe these functions of accumbens DA, including "behavioral activation", "behavioral reactivity", "motivational arousal", and. "psychomotor stimulation" (111,167,187-191,207,208, 1 248-250). In order to capture the integrative nature of these functions of DA, researchers have employed terms such as "sensorimotor" and "limbic-motor" (18,160,161, 188,189,220,256-258). I| A r/ecessary part of this rejection of the motor/reward !1 dichotomy must also be a revision of our understanding of the motor functions of nucleus accumbens. Considerable i, evidence indicates that locomotor activity is reduced by II accumbens DA depletions (44,125-127,246). Yet it should be emphasized that nucleus accumbens DA neurons are several synapses removed from the actual production of ~motor acts; indeed, they are ill suited for the task of directly controlling motor output, and there is little evidence that they are phasically active in specific relation to particular motor acts (191). It has been shown that accumbens lesions did not alter the .motor pattern of locomotion (236) and that accumbens DA depletions did not affect food intake or food handling (197). Interference with accumbens DA does not produce paralysis, and does not lead to the severe motor deficits produced by lateral stdatal DA depletions. The behavioral functions of accumbens differ from those of neostriatum, and these differences probably reflect the functional heterogeneity of striatal subregions as well as the hierarchical organization of frontal lobe and basal ganglia motor circuits (189,196). Nevertheless, accumbens DA depletions produce motor slowing, and appear to make the animal less behaviorally reactive to stimuli, so that higher levels of stimulation are necessary for instigating vigorous instrumental behaviors (43). This would indicate that nucleus accumbens DA is involved in modulating behavioral responsiveness to motivational stimuli, which represents ~in aspect of sensorimotor function. Accumbens DA appears to be involved in the regulation of motor activities but not the specific execution of motor acts. Release of DA in accumbens may prepare organisms for movement, but other structures more directly participate in the execution of motor acts ( 189,191). Depletions of accum- bens DA alter the temporal characteristics of movement, the probability of spontaneous movement being generated, and the ability of stimuli to elicit movement. Accumbens DA depletions do not irreversibly impair the ability to lever press or climb barriers to obtain food, but instead set constraints upon which instrumental response is selected. The general effect of accumbens DA depletions is to alter the relative allocation of responses with different kinetic requirements, such that behavior is biased in the direction of lower effort alternatives. Thus, accumbens DA participates in the process of enabling animals to overcome the obstacles that separate them from significant stimuli such as food. In economic terms, nucleus accumbens DA is important for the relative inelasticity of demand for food reinforcers. Nucleus accumbens DA may participate in the motor planning functions of prefrontal and premotor cortices (43), and may also be involved in modulating some of the conditioned and unconditioned excitatory properties of stimuli (93). DA in nucleus accumbens is important for responding to con- ditioned stimuli, and for responding to stimuli that are spatially and temporally distant from the organism. Are these functions purely sensory, purely motor, or exclusively motivational? Without further qualifications, such terms are not sufficient for describing the behavioral functions of accumbens DA. Perhaps it is better to move towards a different set of terms and embrace a more complex view of the various processes involved in behavioral regulation. ACKNOWLEDGEMENTS Many thanks to K. Sabol, J. Richards, D. Neill, S. Fowler and G. Heyman for their helpful discussions. REFERENCES 1. Abercrombie, E. A.; Keefe, K. A.; DiFdschia, D. A.; Zigmond, M. J., Differential effect of stress on in vivo dopamine release in st_datum, nucleus accumbens and medial frontal cortex. J. Neurochem. 52:1655-1658; 1989. 2. Acquas, E.; Carboni, E.; Leone. P.; Di Chiarra, G., SCH 23390 blocks drug-conditioned place-preference and place aversion: Anhedonia (lack of reward) or apathy (lack of motivation) after dopamine receptor blockade. Psychopharmacol. 99:151-155; 1989. 3. Ader, R.; Clink, D. W., Effects of chlorpromazine on the acquisition and extinction of an avoidance response in the rat..I. Pharmacol. Exp. Ther. 121:144-148; 1957. 4. Agmo, A.; Galvan, A.; Talamantes, B.. Reward and reinforcement produced by drinking sucrose: Two processes that may depend on different neurotransmitters. Pharmacol. Biochem. Behav. 52:403- 414; 1995. 5. Ahlenius; S. Pharmacological evaluation of new antipsychotic drugs. In: Witlner, P., ed., Behavioral Models in Psychopharmacology. Cambridge, England: Cambridge University Press, 1990. 6. Allison J. Economics and operant conditioning. In: Harzem P, Zeiler, M. D., eds., Predictability, Correlation and Contiguity. New York: John Wiley and Sons;1981: 321-353. 7. Allison, J., Response deprivation, reinforcement, and economics. J. Exp. Anal. Behav. 60:129-140; 1993. 8. Anisman, H.; Remington, G.; Sklar, L. S., Effect of inescapable shock on subsequent escape performance: Catecholaminergic and chotinergie mediation of response initiation and maintenance. Psychopharmacol. 61 : 107-124; 1979. 9. Anisman, H.; Corradini, A.; Tombaugh, T. N.; Zacharko, R. M., Avoidance performance, cue and response-choice discrimination after neuroleptic treatment. Pharmacol. Biochem. Behav. 17 1245-1249; 1982. o

354 SALAMONE, COUSINS AND*S~NYDE1~, 10. Asin, K. E.; Fibiger, H. C., Force requirements in lever-pressing and responding after haloperidol. Pharmacol. Biochem. Behav. 20:323- 326; 1984. 11. Bakshi, V. P.; Kelley, A. E., Dopaminergic regulation of feeding behavior: I., Differential effects of haloperidol microinjection in three striatal subregions. Psychobiol. 19:223-232; 1991. 12. Baum, W. M, On two types of deviation from the matching law: bias and undermatching. J. Exp. Anal. Behav. 22:231-242; 1974. 13. Beatty, J. Principles of Behavioral Neuroscience. Madison, WI: Brown and Benchmark; 1995. 14. Beninger, R. J,; Mason, S. T.; Phillips, A. G.; Fibiger, H. C., The use of conditioned suppression to evaluate the nature of neuroleptic- induced avoidance deficits. J. Pharmae. Exp. Ther. 213:623-627; 1980. 15. Beninger, R. J.; Ranaldi, R., Microinjections of flupenthixol into the candate-putamen but not the nucleus accumbens, amygdala or frontal cortex of rats produce intra-session declines in food-rewarded oper- ant responding. Behav. Brain Res. 55:203-212; 1993. 16. Berridge, K. C.; Venier, I. L.; Robinson, T. E., Taste reactivity analysis of 6-hydroxydopamine-induced aphagia: implications for arousal and anhedonia hypotheses of dopamine function. Behav. Neurosci. 103:36-45; 1989. 17. Bindra, D., How adaptive behavior is produced: a perceptual- motivational alternative to response-reinforcement. Behav. Brain Sci. 1:41-91; 1978. 18. Blackburn, J. R.; Pfaus, J. G.; Phillips, A. G., Dopamine functions in appetitive and defensive behaviours. Prog. Neurobiol. 39:247-279; 1992. 19. Blackburn, J. R.; Phillips, A. G.; Fibiger, H. C., Dopamine and preparatory behavior:. I effects of pimozida.. Behav. Neurnsei. 101:352-360; 1987. 20. Blackburn, J. R.; Phillips, A. G., Enhancement of freezing behavior by metoclopramide: implications for neuroleptie-indueed avoidance deficits. Pharmacol. Biochem. Behav. 35:685-691; 1990. 21. Bowers, W.; Hamilton, M.; Zacharcho, R. M.; Anisman, H., Differ- ential effects of pimozide on response-rate and choice accuracy in a self-stimulation paradigm in mice. Pharrnacol. Biochem. Behav. 22:521-526; 1985. 22. Bozarth, M. A.; Wise, R. A., Involvement of the ventral tegmental dopamine system in opioid and psychomotor stimulant reinforcement. Life Sci. 28:551-555; 198l. 23. Bozarth, M. A.; Wise, R. A., Heroin reward is dependent on a dopaminergie substrate. Life Sci. 29:1881-1886; 1981. 24. "Broekkamp, C. L.; van Dongen, P. A.; van Rossum, Neostriatal involvement in reinforcement and motivation. In: Cools, A. R., Lohman, A. M., van den Berken, J. H., eds, Psychobiology of the Striatum. Amsterdam: Elsevier North Holland; 1977. 25. Cador, M.; Robbins, T. W.; Everitt, B. J., Involvement of the amygdala in stimulus-reward associations: interactions with the ventral striatum. Neurosci. 30:77-86; 1989. 26. Cador, M.; Taylor, J. R.; Robbins, T. W., Potentiation of the effects of reward-related stimuli by dopaminergic-dependent mechanisms in the nucleus accumbens. Psychopharrnacol. 104:377-385; 1991. 27. Carlson, N. Foundations of Physiological Psychology. Needham Hights, MA: Allyn an Bacon, 1995;. 28. Carr, G. D.; White, N. M., The relationship between stereotypy and memory improvement produced by amphetamine. Psychopharmacol. 82:203-209; 1984. 29. Cenci, M. A.; Kalen, P.; Mandel, R. J.; Bjorklund, A., Regional differences in the regulation of dopamine and noradrenaline release in medial frontal cortex, nucleus accumbens and caudate- putamen: a microdialysis study in the rat. Brain Res. 581:217-228; 1992. 30. Cenci, M. A.; Kaien, P.; Duan, W. M.; Bjorkland, A., Transmitter release from transplants of fetal ventral mesencephalonor locus coeruleus in the rat frontal cortex and nucleus accumbens: effects of pharmacological and behaviorally activating stimuli. Brain Res. 641:225-248; 1994. 31. Chert, J.; Paredes, W.; Li, J.; Smith, D.; Lowison, J.; Gardner, E. L., 9-tetrahydrocannabinol produces naloxone blockable enhancement of presynaptie basal dopamine efflux in nucleus accumbens of concious, freely-moving rats as measured by intracerebral microdialysis. Psychopharrnacol. 102:156-162; 1990. 32. Cliftun, P. G.; Rusk, [. N.; Cooper, S. J., Effects of dopamine D1 and dopamine D2 antagonists on the free feeding and drinking patterns of rats. Behav. Neurosci. 105:272-281; 1991. 33. Cofer, C. N.; Appley, M. H. Motivation: Theory and Research. New York: John Wiley & Sons; 1964. 34. Collier, G. H.; Jennings, W., Work as a determinant of instrumental performance. J. Comp. Physiol. Psychol. 68:659-662; 1969. 35. Collier, G. H.; Johnson, D. F.; Hill, W. L.; Kaufman, L. W., The economics of the law of effect. J. Exp. Anal. Behav. 46:113-136; 1986. 36. Collins, R. C., Kindling of neuroanatomic pathways during recurrent focal penicillin seizures. Brain Res. 150:503-517; 1978. 37. Colwill, R. M.; Rescorla, R. A. Associative s/.ructures in instrumental learning. In Bower G. H. ed. The Psychology of Learning and Motivation, New York, Academic Press;1986:55-104. 38. Cook, L.; Catania, C., Effects of drugs on escape and avoidance behavior. Fed. Proc. 23:818-835; 1964. 39. Cook, L.; Weidiey, E., Behavioral effects of some psychopharma- cological agents. Ann. NY Acad. Sci. 66:740-752; 1957. 40. Cooper, S. J.; Francis, J., A microstructural analysis of the effects of presatiation on feeding behavior in the rat. Physiol. Behav. 53:413- 419; 1993. 41. Cooper, S. J.; Sweeny, K. F., Effects of spiperone alone and in combination with anorectic agents on feeding parameters in the rat. Neuropharmacol. 19:997-1003; 1980. 42. Corradini, A.; Tombaugh, T.; Anisman, H., Effects of pimozide on escape and discrimination performance in a water-escape task. Behav. Neurosci. 98:96-106; 1984. 43. Cousins, M. S.; Atherton, A.; Turner, L.; Salamone, J. D.; Nucleus accumbens dopamine depletions alter relative response allocation in a T-maze cost/benefit task. Behav. Brain Res. 74:189-197; 1996. 44. Cousins, M. S.; Sokolowski, J. D.; Salamone, J. D., Different effects of nucleus accumbens and ventrolateral striatal dopamine depletions on instrumental response selection in the rat. Pharmacol..Biochem Behav. 46:943-951; 1993. 45. Cousins, M. S.; Salamone, J. D., Nucleus accumbens dopamine depletions in rats affect relative response allocation in a novel cost/ benefit paradigm. Pharmacol. Biochem. Behav. 49:85-91; 1994. 46. Cousi~as, M. S.; Salamone, J. D., Involvement of ventrolateral striatal dopamine in movement initiation and execution: A microdialysis and behavioral investigation. Neurosci. 70:849-859; 1996. 47. Cousins, M. S.; Wei, W.; Salamone, J. D., Pharmacological char- acterization of performance on a concurrent lever pressing/feeding choice procedure: effects of dopamine antagonist, chulinomimetic, sedative and stimulant drugs. Psychopharmacol. 116:529-537; 1994. 48. Crow, T. J., Catecholamine-containing neurones and electrical self- stimulation: a review of some data. Psychol. Med. 2:414-421; 1972. 49. Cunningham, C. L.; Malott, D. H.; Dickinson, S. D.; Risinger, F. O., Haloperidol does not alter expression of ethanol-induced conditioned place prefernce. Behav. Brain Res. 50:1-5; 1990. 50. D'Angio, M. B.; Serrano, A.; Rivy, J. P.; Scatton, B., Tail-pinch stress increases extracellular DOPAC levels (as measured by in vivo vol- tammetry) in rat nucleus accumbens but not frontal cortex: Antagon- ism by diazepam and zolpidem. Brain Res. 409:169-174; 1987. 51. D'Angio, M. B.; Serrano, A.; Scatton, B., Mesocorticolimbic dopaminergie systems and emotional states. J. Neurosci. Meth. 34:135-142; 1990. 52. Davidson, A. B.; Weidley, E., Differential effects of neuroleptic and other psychotropic agents on aquisition of avoidance in rats. Life Sci. 18:1279-1284; 1976. 53. Deutch, A. Y.; Cameron, D. S., Pharmacological characterization of dopamine systems in the nucleus accumbens core and shell. Neurosci. 46:49-56; 1992. 54. Deutch, A. Y.; Tam, S. Y.; Roth, R. H., Footshock and conditioned stress increase 3,4-dihydroxyphenylacetic acid (DOPAC) in the ventral tegmental area but not substantia nigra. Brain Res. 333:143-146; 1985. 55. Di Chiara, G.; Imperato, A., Preferential stimulation of dopamine release in the nucleus accumbens by opiates, alcohol and barbitu- rates: studies with transcerebral dialysis in freely moving rats. Ann. NY Acad. Sci. 473:367-381; 1986. 56. DiStain, G.; Sandner, G., Conditioned place aversion produced by FG 7142 is attenuated by haloperidol. Psychopharmacol. 99:176- 180; 1989. 57. Duffy, E. Activation and Behavior. New York: John Wiley & Sons; t963. 58. Dunn, A., Stress-related activation of cerebral dopaminergic systems. Ann. NY Acad. Sci. 537:124-137; 1988. i.

~cd ess ~bic eth. Sci. n of hell. ,reed , the Res. mine ~---~ ~bitu- Ann. 8~HAVIORAL FUNCTIONS OF NUCLEUS ACCUMBENS DOPAMINE 355 59. Dunn,. A.; File, S. E., Cold restrain alters dopamine metabolism in frontal cortex, nucleus accumbens and neostriatum. Physiol. Behav. 31:511-513; 1983. 60. Dunnett, S. B.; Iversen, S. D., Regulatory impairments following selective 6-OHDA lesions of the neostriatum. Behav. Brain Res. 4:195-202; 1982. 61. Edmonds, P. E.; Gallistel C. R. Parametric analysis of brain stimulation reward in the rot: effect of performance variavles on the reward summation function. J. Comp. Physiol. Psychol. 87:876- 883; 1974. 62. Ettenberg, A.; Carlisle, H. J., Neuroleptie-induced deficits in operant responding for temperature reinforcement. Pharmacol. Biochem. Behav. 22:761-767; 1985. 63. Ettenberg, A.; Koob, G. F.; Bloom, F., Response artifact in the .measurement of neurolepdc-induced anhcdoni& Science 209:357- 359; 1981. 64. Ettenberg, A.; Petfit, H. O.; Bloom, F. E.; Koob, G.; Heroin and cocaine self-administration in rats: mediation by separate systems. Psychopharmacol. 78:204-209; 1982. 65. Evenden, J. L.; Robbins, T.W., Dissociable effects old-amphetamine, chlordiazepoxide and alpha-flupenthixol on choice and rote measures of reinforcement in the rat. Psychopharmacol. 79:180-186; 1983. . 66. Evenden, L L.; Robbins, T. W., Effects of unilateral 6-hydroxy dopaminc lesions of the caudate-putamen on skilled forelimb use in the rat. Behav. Brain Res. 14:61-68; 1984. 67. Everitt, B. J., Sexual motivation: A neural and behavioral analysis of the mechanisms underlying appetitive and copulatory responses of male rats. Neurosei. Biobehav. Rev. 14:217-232; 1990. amygdala and ventral striamm in stimulus-reward association: studies assessing a second-order schedule of sexual reinforcement. Neurosci. 30:63-75; 1989. 69. Everitt, B. J.; Morris, K. A.; O'Brien, A.; Robbins, T. W., The basolateral amygdala-ventral striatal system and conditioned place preference: Further evidence of limbic-striatal interactions under- lying reward-related processes. Neurosci. 42:1-18; 1991. 70. Fada, F.; Argiolas, A.; Melis, M. R.; Tissari, A. H.; Onaii, P. C.; Gessa, G. L., Stress-induced increase in 3,4-dihydroxyphenylacetie acid (DOPAC) levels in the cerebral cortex and in nucleus aecum- bens: reversal by diazepam. Life Sei. 23:2219-2224; 1978. 71. Faustman, W. 04 Fowler, S. C., Use of operant response duration to distinguish effects of haloperidol from non-reward. Pharmacol. Biochem. Behav. 15:327-329; 1981. 72. Faustman, W. O.; Fowler, S. C., An examination of methodological refinements, clozapine and fluphenazine in the anhedonia paradigm. Pharmacol. Binchem. Bchav. 17:987-993; 1982. 73. Fibiger, H. C.; Carter, D. A.; Phillips, A. G., Decreased intracranial self-stimulation after neuroleptics or 6-hydroxydopaminc: evidence for mediation by reward deficits rather than by reduced reward. Psychopharmacol. 47:21-27; 1976. 74. Finlay, J. M.; Damsma, G.; Fibiger, J. C., Banzodiazepine-induced decreases in extracellular concentrations of dopamine in the nucleus accumbens after acute and repeated administration. Psychopharmacol. 106:202-208; 1992. 75. Fouriezos, G.; Bielajew, C.; Pagotto, W., Task difficulty increases threshold of rewarding brain stimulation. Behav. Brain Res. 37:1-7; 1990. 76. Fouriezos, G.; Wise, R. A., Pimozide-induced extinction of intracra- nial self-stimulation: Response patterns rule out moto performance deficits. Brain Res. 103:377-380; 1980. 77. Fowler, S. C. Neurolepdcs produce within-session decrements: facts and theories. Drug Dee. Res. 1990. 78. Fowler, S. C.; Das, S., Haloperidol-induced decrements in force and duration of rats tongue movements during licking are attenuated by concomitant anticholinergic treatment. Pharmacol. Biochem. Behav. 49:813-817; 1994. 79. Fowler, S. C.; LaCerra, M. M.; Ettenberg, A., Effects of haloperidol on the biophysical characteristics of operant responding: Implica- tions for motor and reinforcement processes. Pharmacol Biochem Behav. 25:791-796; 1986. 80. Fowler, S. C.; Mortell, C., Low doses of haloperidol interfere with rat tongue extensions during licking: a quantitative analysis. Behav. Neurosci. 106:1-9; 1992. 81. Frank, R.; Williams. H., Both response effort and current intensity affect self-stimulation train duration thresholds. Pharmacol Biochem Behav. 22:527-530; 1985. 82. Gaddy, J. R.; Neill, D. B., Differential behavioral changes following intrastiatal applications of 6-hydroxydopamine. Brain Res. 119:439- 446; 1977. 83. Garmon, K. N.; Smith, H. V.; Tiemey, K. J., Effects of procure- ment cost on food consumption in rats. Physiol Behav. 31:331- 337; 1983. 84. German, D. C.; Bowden, D. M., Catecholamine systems as the neural substrate for intraeranial self-stimulation: a hypothesis. Brain Res. 73:381-419; 1974. 85. Gerrits, M.; Ramsey, N. F.; Wolterink, G.; van Ree, J. M., Lack of evidence for an involvement of nucleus accumbens dopamine D1 receptors in the initiation of heroin self-adm/nistration. Psychopharmacol. 114:486-494; 1994. 86. Glickman, S. E.; Schiff, B. B., A biological theory of reinforcement. Psychol. Rev. 74:81-109; 1967. 87. Gold, M. S.; Miller, N. S.; Jonas, J. M. Cocaine (and crack): Neurobiol- ogy. In Lowinson, J. H. and P. Ruiz, Robert MAilman eds. Substance Abuse. Baltimore: William and Wilkins; 1992: 222-235.; 88. Gramling, S. E.; Fowler, S. C., Effects of neuroleptics on rate and duration of operant versus reflexive licking in rats. Pharmacol. Biochem. Behav. 22:541-545; 1985. 89. Gramling, S. E.; Fowler, S. C.; Collins, K. R., Some effects of pimozide on nondepdved rats licking sucrose solutions in an arahedonia paradigm. Pharmacol. Biochem. Behav. 21:617-624; 1984. 90. Gramling, S. E.; Fowler, S. C.; Tizzano, I. P., Some effects of pimozide on nondeprived rats' lever pressing maintained by a sucrose reward in an anhedonia paradigm. Pharrnacol. Biochem. Bchav. 27:67-72; 1987. 91. Green, D. M.; Swetts, J. A. Signal Detection Theory and Psycho- physics. New York Wiley; 1966. 92. Gunne, L. M.; Anggard, E.; Jonsson, L. E., Clinical trials with amphetamine-blocking drugs. Psychiat, Neurol. Neuro. 75:225- 226; 1972. 93. Harvey, J. A.; Gormezano, I., Effects ofhaloperidol and pimozide on classical conditioning of the rabbit nictitating membrane response. J. Pharmacol. Exp. Ther. 218:712-719; 1981. 94. Hemandez, L.; Hoebel, B. G., Food reward and cocaine increase extracellular dopamine in the nucleus accumbens as measured by ,microdialysis. Life Sci. 42:1705-1712; 1988. 95. Herrnstein, R. J., Formal properties of the matching law. J. Exp. Anal. Behav. 21:159-164; 1974. 96. Hcyman, G. M., A parametric evaluation of hedonic and motodc effects of drugs: pimozide and amphetamine. J. Exp. Anal. Behav. 40:113-122; 1983. 97. Heyman, G. M.; Kirmie, D. L.; Selden, L. S., Chlorpromazine and pimozide alter reinforcement efficacy and motor performance. Psychopharrnacol. 88:346-353; 1986. 98. Hcyman, G. M.; Monaghan, M. M., Effects of changes in response requirement and deprivation on the parameters of the matching law equation: new data and review. J. Exp. Psychol. An. Behav. Proc. 13:384-394; 1987. 99. Hilgard, E. R. Marquis, D. G. Conditioning and Learning. Appleton- Century Co., New York; 1940.; 100. Hoebel, B. G.; Monaco, A. P.; Hemandez, L.; Aulisi, E. F.; Stanley, B. G.; Leonard, L., Self-injection of amphetamine directly into the brain. Psychopharmacol. 81:158-163; 1983. 101. Hoffman, D. C.; Beninger, R. J., The effects of selective dopamine D1 or D2 receptor antagonists on the establishment of agonist- induced place conditioning in rats. Pharmacol. Biochem. Behav. 33:273-279; 1989. 102. Hsiao, S.; Chcn, B. H., Complex response competition and dopaminc blocking: Choosing of high cost sucros~ solution vs. low cost water solution in rats. Chin. J. Physiol. 38:99-109; 1995. 103. Hsiao, S.; Smith, G. P., Raclopride reduces sucrose preference in rats. Pharmacol. Biochem. Behav. 50 (1):121-125; 1995. 104. Hull, E. M.; Weber, M. S.; Eaton, R. C.; Dua, R.; Markowski, V. P.; Lumley, L.; Moses, J., Dopaminc receptors in the ventral tcgmental area affect motor, but not motivational or reflexive, components of copulation in male rats.~-~'cain Res. 554:72-76; 199 I. 105. Hursh, S. R., Behavioral Economics. J. Exp. Anal. Behav. 42:435- 452; 1984. 106. Hursh, S. R.; Raslear, T. G.; Shurtleff, D.; Bauman, R.; Simmons, L., A cost-benefit .analysis of demand for food. J Exp Anal Behav. 30:419-440; 1988. 107. Huston, J. P.; Steiner. H.; Weiler, H.; Morgan, S.; Schwarting, R. K.W., The basal ganglia-orofacial system: studies on neuro-beha-

356 SALAMONE, COUSINS AND SNYDER vioral plasticity and sensory-motor tuning. Neurosci. Biobehav. Rev. 14:433-466; 1990. 108. lmperato, A.; Angelucei, L.; Casolini, P.; Zocchi, A.; Puglisi- Allegra, S., Repeated stressful experiences differently affect limbie dopaminc release during and following stress. Brain Res. 577:194- 199; 1992. 109. Imperato, A.; Puglisi-Allegra, S.; Casoline, P.; Angclucci, L., Changes in brain dopamine and acetylcholine release during and following stress are independent of the pituitary-adrenocortical axis. Brain Res. 538:111-117; 1991. 110. Jicha, G. A.; Salamone, J. D., Vacuous jaw movements and feeding deficits in rats with ventrolateral striatal dopamine depletions: possible relation to parkinsonian symptoms. J Neurosci. 11:3822- 3829; 1991. 11 I. Johnson, P. L.; Stellar, J. R., Effects of accumbens DALA micro- injections on brain stimulation reward and behavioral activation in intact and 6-OHDA treated rats. Psyehopharmacol. 114 (4):665-671; 1994. 112. Jones, D. L.; Mogenson, G. L., Oral motor performance following control dopamine receptor blockade. Eur. J. Pharmacol. 59:11-21; 1979. 113. Joseph, M. H.; Hedges, H., Lever pressing for food reward and changes in dopamine turnover and uric acid in rat caudate and nucleus accumbens studied chronically by in vivo voltammera3,. J. Neurosci. Meth. 34:143-149; 1990. 114. Kaufman, L. W., Foraging cost and meal patterns in ferrets. Physiol Behav. 25:139-141; 1980. 115. Keefe, K. A.; Salamone, J. D.; Zigmond, M. J.; Stricker, E. M., Paradoxical kinesia in Parkinsonism is not caused by dopamine release: studies in an animal models. Arch. Neurol. 46:1070-1075; 1989. 116. Kelleher, R. T.; Morse, W. H., Escape behavior and punished behavior. Fed. Prec. 23:808-817; 1964. 117. Kelley, A. E.; Dells, J. M., Dopamine and conditioned reinforcement II: contrasting effects of amphetamine mierninjeetion into the nucleus accumbens with peptide mieroinjection into the ventral tegmental area. Psychopharmacol. 103:197-203; 1991. 118. Kelley, A. E.; Stinus, L., Disappearance of hoarding behavior after 6- hydroxydopamine lesions of the mesolimbic dopamine neurons and its reinstatement with L-DOPA. Behav. Neurosci. 99:531-5435; 1985. 119. Killeen, P., On the temporal control of behavior. Psychol. Rev. 82:g9-115; 1975. 120. Killeen, P. Incentive theory. In: Bernstein D. ed. Response structure and organization. University of Nebraska Press, Lincoln Nebraska; 1981. 121. Killeen, P.; Hanson, S.; Osbourne, S., Arousal: its genesis and manifestation as response rate. Psychol. Rev. 85:571-581; 1978. 122. Kirkpatrick, M. A.; Fowler, S. C., Force-proportional reinforcement: pimozide does not reduce rats' emmission of higher forces for sweeter rewards. Pharmacol. Biocbem. Behav. 32:499-504; 1989. 123. Kiyatkin, E. A.; Gratton, A., Electrochemical monitoring of extra- cellular dopamine in nucleus accumbens of rats lever pressing for food. Brain Res. 652:225-234; 1994. 124. Kiyatkin, E. A.; Wise, R. A.; Gratton, A., Drug- and behavior- associated changes in dopamine-related electrochemical signals during intravenous heroin self-administration in rats. Synapse 14:60-72; 1993. 125. Koob, G. F.; Riley, S. J.; Smith, S. C.; Robbins, T. W., Effects of 6- hydroxydopamine lesions of the nucleus accumbens septi and olfac- tory tubercle on feeding, locomotor activity, and amphetamine anorexia in the rat. J. Comp. Physiol. Psychol. 92:917-927; 1978. 126. Koob, G. F.; Stinus, L.; Le Meal, M., Hyperactivity and hypoactivity produced by lesions to the mesolimbic dopamine system. Behav. Bmin Res. 3:341-359; 1981. 127. Koob, G. F.; Swerdlow, N. R., The functional output of the mesolimbic dopamine system. Ann. N. Y. Acad. Sci. 537:216-227; 1988. 128. Kosobud. A. E.; Harris, G. C.; Chapin, J. K., Behavioral associations of neuronal activity in the ventral tegmental area of the rat. J. Neurosci. 14:7117-7129; 1994. 129. Krebs, J. R. Optimal foraging: Decision rules for predators. In Krebs, J. R., Davies, W. B., eds. Behavioral Ecology. Sunderland, MA: Sinaue Associates; 1978. 130. Krebs, J. R., Optimal foraging: theory and experiment. Nature 268:583-584; 1977. 131. Kuo, Z. Y. The dynamics of behavior development: An epigenetic view. New York, Random House; 1967. 132. Lea, S. E. G., The psychology and economies of demand. Psychol. Bull. 85:441-466; 1978. 133. Liao, R~ M.; Fowler, S. C., Halopeddol produces within-session increments in operant response duration in rats. Pharmacol. Biochem. Behav. 26:1999-2010; 1990. 134. Lidsky, T. I4 Buehwald, N. A.; Manetto, C.; Schneider, J. S.. A consideration of sensory factors involved in the motor functions of the basal ganglia. Brain Res. Rev. 9:133-146; 1985. 135. Lippa, A. S.; Antleman, S.; Fisher, A.; Canfield, D., Neurochemical mediation of reward: a significant role for dopamine. P.harmacol. Biochem. Behav. 1:25-28; 1973. 136. Ljungberg, T., Blockade by neuroleptics of water intake and operant responding in the rat: anhedonia, motor deficit or both?. Pharmacol. Biochem. Behav. 27:341-350; 1987. 137. Ljungberg, T., Scopolamine reverses haloperidol-attenuated lever pressing for water but not halopeddol-attenuated water intake in the rat. Pharmacol. Biochem. Behav. 29:205-208; 1988. 138. Ljungberg, T., Differential attenuation of water intake and water- rewarded operant responding by repeated administration of haloper- idol and SCH 23390 in the rat. Pharmacol. Biochem. Behav. 35:111- 115; 1990. 139. Ljungberg, T.; Apicella, P.; Schultz, W., Responses of monkey dopamine neurons during learning of behavioral reactions. J. Neurophysiol. 67:145-163; 1992. 140. Lynch, M. R.; Carey, R. J., Environmental stimulation promotes recovery from haloperidol-induced extinction of open field behavior in rats. Psychopharmacol. 92:206-209; 1987. 141. Mackintosh, N. J., Limits on reinterpreting instrumental conditioning in terms of classical conditioning. Behav. Brain Sci. 1:67; 1978. 142. Mackinstosh, N. J. Conditioning and Associative Learning. NY, Oxford University Press; 1983. 143. Maldonado-Irizarry, C. S.; Kelley, A. E., Differential behavioral , effects following microinjection of an NMDA antagonist into nucleus aecumbens subregions. Psychopharmacol. 116:65-72; 1994. 144. Mark, G. P.; Schwarz, D. H.; Hernandez, L.; West, H. L.; Hoebel, B. G. Application of microdialysis to the study of motivation and conditioning: Measurements of dopamine and serotonin in freely- behaving rats. In Robinson, T. E., Justice, J. B., Microdialysis in the Neuroseiences; 1991:369-385. 145. Marshall, J. F.; Levitan, D.; Stricker, E. M., Activation-induced restoration of sensorimotor functions in rats with dopamine-depleting brain lesions. J. Comp. Physiol. Psychol. 90:536-546; 1976. 146. Marshall, J. F.; Richardson, J. S.; Teitelbaum, P., Nigrostriatal bundle damage and the lateral hypothalamic syndrome. J. Comp. Physiol. Psychol. 87:808-830; 1974. 147. Martin-Iverson, M. T.; Wilke, D.; Fibiger, H. C., Effect of haloper- idol and d-amphetamine on perceived quantitiy of food and tones. Psyehopharmacol. 93:374-381; 1987. 148. Mason, S. T.; Beninger, R. J.; Fibiger, H. C.; Phillips, A. G., pimozide- induced suppression of responding: evidence against a block of food reward. Pharmacol. Biochem. Behav. 12:917-923; 1980. 149. McCullough, L. D.; Cousins, M. S.; Salamone, J. D., The role of nucleus accumbens dopamine in responding on a continuous rein- forcement operant schedule: a neurochemical and Behavioral study. Pharmac. Biochem. Behav. 46:581-586; 1993. 150. McCullough, L. D.; Salamone, J. D., Anxiogenie drugs beta-CCE and FG 7142 increase extracellular dopamine levels in the nucleus accumbens. Psychopharmacol. 109:379-382; 1992. 151. McCullough, L. D.; Salamone, J. D., Involvement of nucleus accumbens dopamine in" the motor activity induced by periodic food presentation: a microdialysis and behavioral st.udy. Brain Res. 592:29-36; 1992. 152. McCullough, L. D.; Sokolowsi, J. D.; Salamone, J. D.. A neuro- chemical and behavioral investigation of the involvement of nucleus accumbens dopamine in instrumental avoidance. Neurosci. 52:919- 925; 1993. 153. McDowell, J. J.; Kessell, R., A multivariate rate equation for variable- interval performance. J. Exp. Anal. Behav. 31:267-283; 1979. 154. McGeorge, A. J.; Faull, R. L.M.. The organization of the projection from the cerebral cortex to the striatum in the rat. Neuroscience 29:503-537; 1989. 155. Wleehl, P. E., On the circularity of the law of effect. Psychol. Bull. 47:52-75; 1950. 156. Meisel, R. L.; Camp, D. M.; Robinson. T. E., A microdialysis study

!I ~ing 169. ~dle ,iol. fles. 171. ood 172. ~" of ein- 173. "dY" I ~ 174. 'CE leus l-lj leus 175. ~dic. Res. 176. BEHAVIORAL FUNCTIONS OF NUCLEUS ACCUMBENS DOPAMINE 177. tudy 357 of ventral striatal dopamine during sexual behavior in female Syrian hamsters. Behav. Brain Res. 55:151-157; 1993. Meredith, G. E.; Blank, B.; Groenewegen, H. J., The distribution and compartmental organization of the cholinergic neurons in the nucleus accumbens of the rat. Neurosci. 31:327-345; 1989. Mirenowicz, J.; Schultz, W., ImpoRanee of unpredictability for reward responses in primate dopamine neurons. J. Neurophys. 72 (2):1024-1027; 1994. Mitchell, J. B.; Gratton, A., Opiod modulation and sensitization of dopamine re/ease elicited by sexual/y relevant sdmuli: a high speed chronoamperometrie study in freely behaving rats. Brain Res. 551:20-27; 1991. Mogenson, 'G.; Jones, D.; Yim, C. Y., From motivation to action: Functional interface between the limbie system and the motor system. Prog Neurobiol. 14:69-97; 1980. Mogenson, G. J.; Yang, C. R.; Yim, C. Y., Influence of dopamine limbic inputs to the nucleus accumbens. Ann. NY Acad. Sci. 537:86- i00; 1988. Muscat, R.; Willner, P., Effects of dopamine receptor antagonists on sucrose consumption and preference. Psyehopharmacol. 99:98-102; 1989. Neill, D. B.; Justice, J. B. An hypothesis for a behavioral function of. dopaminergie transmission in nucleus accumbens. In: Chronister, R. B., Defrance, J. F., eds., The Neurobiology of the Nucleus Accum- bens. Brunswick, Canada: Hue Institute; 1981. Niemegeers, C. J. E.; Verbmggen, F. J.; Janssen, P. A.J., The influence of various neuroleptic drugs on shock avoidance respond- ing rats. Psychopharmacol. 16:161-174; 1969. Niemegeers, C. J. E.; Verbruggen, F. J.; Janssen, P. A.J., The influence of various neuroleptie drugs on shock avoidance respond- ing in rats. Psyehopharmaeol. 16:175-182; 1969. Nishino, H.; Ono, T.; Muramoto, K.; Fukuda, M.; Sasaki, K., Neuronal activity in the ventral tegmental area (VTA) during motivated bar press feeding in the monkey. Brain Res. 413:302- 313; 1987. Parada, M. A.; Parada, M. P.D.; Hoebel, B. G., Rats self-inject a dopamine antagonist in the lateral hypothalamus where it acts to increase extracellular dopamine in the nucleus accumbens. Pharma- col. Biochem. Behav. 52 (1):179-187; 1995. Phillips, A. G.; Fibiger, H. C., Decreased resistance to extinction after halopeddol: Implications for the role of dopamine in reinforcement. Pharmacol. Biochem. Behav. 10:751-761; 1979. Phillips, G. D.; Robbins, T. W.; Everitt, B. J., Bilateral intra- accumbens self-administration of d-amphetamine: Antagonism with intra-accumbens SCH-23390 and sulpiride. Psychopharmacol. 114:477-485; 1994. Packard, M. G.; Hirsh, R.; White, N. M., Differential effects of fornix and caudate nucleus lesions on two radial maze tasks: evidence for multiple memory systems. J. Neurosci. 9:1465-1472; 1991. Porter, J. H.; Villanueva, H. F., Assessment of pimozide's motor and hedonic effects on operant behavior in rats. Pharmacol. Biochem. Behav. 31:779-786; 1989. Posluns, D., An analysis of chlorpromazine-induced suppression of the avoidance response. Psychopharmacol. 3:361-373; 1962. Premack, D., Toward empirical behavior laws I: positive reinforcement. Psychot. Rev. 66:219-233; 1959. Radhakishun, F. S.; van Ree, J. M.; Westerink, B. H.C., Scheduled eating increases dopamine release in the nucleus accumbens of food- deprived rats as assessed with on-line brain dialysis. Neurosci. Lett. 85:35t-356; 1988. Rashotte, M. E.; Henderson, D., Coping with rising food costs in a closed economy: feeding behavior and nocturnal hypothermia in pigeons. J. Exp. Anal. Behav. 50:441-456; 1988. Rassnick, S.; Stinus, L.; Koob, G. F., The effects of 6- hydroxydo- pamine lesions of the nucleus accumbens and the mesolimbic dopamine system on oral self-administration of ethanol in the rat. Brain Res. 623:16-24; 1993. Rescorla, R. A., The role of information about the response-outcome relation in instrumental learning. J. Exp. Psychol. An. Behav. Prec. 16:262-270; 1990. Richardson, N. R., Change, A. Gratton, N. R., dopamine efflux in nucleus accumbens during operant responding for food: effects of reinforcement value and schedule. Soc. Neumsci. Abs. 20:822; 1994. Roberts, D. C. S.; Corcoran. M. E.; Fibiger, H. C., On the role of ascending catecholaminergic systems in intravenous self-administra- tion of cocaine. Pharmacol. Biochem. Behav. 6:615-620; 1977. 180. Robbins, T. W.; Koob, G. F., Selective disruption of displacement behaviour by lesions of the mesolimbic dopamine system. Nature 285:409-412; 1980. 181. Robinson, T. E.; Becker, J. B.; Young, E. Jk.; Akil, H.; Castenada, E., The effects of footshock stress on regional brain dopamine metabo- lism and pituitary beta-endorphin release in rats previously sensitized to amphetamine. Neuropharmaool. 26:679-691; 1987. 182. Robinson, T. F_,; Berridge, K. C., The neural basis of drug craving: an incentive-sensitization theory of addiction. Brain Res. Rev. 18:247- 291; 1993. 183. Roils, E. T.; Rolls, B. J.; Kelly, P. H.; Shaw, S. G.; Wood, R. J.; Dale, R., The relative attenuation of self-stimulation, eating and drinking produced by dopamine receptor blockade. Psychopharmacol. 38:219-230; 1974. 184. Sabol, K. E.; Neill, D. B.; Wages, S. A.; Church, W. H.; Justice, J. B., Dopamine depletion in a striatal subregion disrupts performance of a skilled motor task in the rat. Brain Res. 335:33-43; 1985. 185. Salamone, J. D., Different effects of haloperidul and extinction on instrumental behaviors. Psychophannacul. 88:18-23; 1986. 186. Salamone, J. D., The actions of neuroleptie drugs on appetitive instrumental behaviors. In Iversen L. L., Iversen S. D., Snyder S. H., eds. Handbook of Psychopharmacology. New York: Plenum Press;1987: 575-608. 187. Salamone, J. D., Dopaminergic involvement in activational aspects of motivation: effects of halopeddol on schedule-induced activity, feeding and foraging in rats. Psychobiol. 16:196-206; 1988. 188. Salamone, J. D., Behavioral pharmacology of dopamine systems: A new synthesis. In: Willner, P., Scheel-Kruger, J., eds., The Meso- limbic Dopamine System: From Motivation to Action. Cambridge, England: Cambridge University Press;1991: 599-613. 189. Salamone, J. D., Complex motor and sensorimotor functions of accumbens and striatal dopamine: Involvement in instrumental behavior processes. Psyehopharmaeol. 107:160-174; 1992. 190. Salamone, J. D., The involvement of nucleus accumbens dopamine in appetitive and aversive motivation. Behav Brain Res. 61:117-133; 1994. 191. Salamone, J. D., The behavioral neurochemistry of motivation: Methodological and conceptual issues in studies of the dynamic activity of nucleus accumbens dopamine. J. Neurosci. Meth. 64:137- 149 1996. 192. Salamone, J. D.; Cousins, M. S.; Bucher, S., Anhedonia or anergia? Effects of haloperidol and nucleus accumbens dopamine depletion on instrumental response selection in a T-maze cost/benefit procedure. Behav. Brain Res. 65:221-229, 1994. 193. Salamone, J. D.; Cousins, M. S.; Maio, C. Champion, M.; Turski, T.; Kovach, J., Different behavioral effects of haloperidol, clozapine and thioridazine in an instrumental lever pressing/feeding procedure. Psychopharmacol. 125:105-112; 1996. 194. Salamone, J. D.; Cousins, M. S.; McCullough, L. D.; Carriero, D. L.; Berkowitz, R. L., Nucleus accumbens dopamine release increases during instrumental lever pressing for food but not food consumption. Pharmacol. Biochem. Behav. 49:25-31; 1994. 195. Salamone, J. D.; Kurth, P. A.; McCullough, L. D.; Sokolowski, J. D., .The effects of nucleus accumbens dopamine depletions on continu- ously reinforced operant responding: contrasts with the effects of extinction. Pharmacol. Biochem. Behav. 50:437-443; 1995. 196. Salamone, J. D.; Kurth, P. A.; McCullot/gh, L. D.; Sokolowski, J. D.; Cousins, M. S., The role of brain dopamine in response initiation: effects of halopeddol and regionally-specific dopamine depletions on the local rate of instrumental responding. Brain Res. 628:218-226; 1993. 197. Salamone, J. D.; Mahan, K.; Rogers, S., Ventrolateral striatal dopamine depletions impair feeding and food handling in rats. Pharmacol. Biochem. Behav. 44:605-610; 1993. 198. Salamone, J. D.; Steinpreis, R. E.; McCullough, L. D.; Smith, P.; Grebel, D.; Mahan, K., Haloperidol and nucleus accumbens dopa- mine depletion suppress lever pressing for food but increase free food consumption in a novel food-choice procedure. Psychopharmacot. 104:515-521; 1991. 199. Salamone, J. D.; Zigmond, M. J.; Stdcker, E. M., Characterization of the impaired feeding behavior in rats given haloperidol or dopamine- depleting brain lesions. Neurosci. 39:17-24; 1990. 200. Samson, H. H.; Tolliver, G. A.; Haraguchi, M.; Kalivas, P. W., Effects of d-amphetamine injected into the nuclues accumbens on ethanol reinforced behavior. Brain Res. Bull. 27:267-27I; 1991.

358 cotJs s AND 201. Sanger, D. J., Response decrement patterns after neurolepfie and non- neuroleptie drags. Psychopharmacol. 89:98-104; 1986. 202. Scatton, B.; D'Angio, M.; Driscoll, P.; Serrano, A., An in vivo voltammetric study of the response of mesocortical and mesoaccum= bens dopaminergic neurons to environmental stimuli in strains of rots with differing levels of emotionality. Ann. NY Acad. Sci. 537:124- 137; 1988. 203. Schneider, J. S., Basal ganglia role in behavior: importance of sensory gating and its relevance to psychiatry. Biol. Psychiat. 19:1693-1710; 1984. 204. Schneider, L. H.; Davis, J. D.; Watson, C. A.; Smith, G. P., Similar effects of raclopride and reduced sucrose concentration on the micmstmcture of sucrose sham feeding. Eur. J. Pharmacol. 186:61-70; 1990. 205. Schneider, L. H.; Gibbs, L; Smith, G. P., D-2 selective receptor antagonists suppress sucrose sham feeding in the rat. Brain Res. Bull. 17:605-61 l; 1986. 206. Schneider, L. H.; Watson, C. A.; Davis, J. D.; Smith, G. P., Microstructurai analysis of the inhibition of sucrose sham feeding by SCH 23390. Appetite 12:236; 1989. 207. Schultz, W., Responses of midbrain dopamine neurons to behavioral trigger stimuli in the monkey. J. Ncurophysiol. 56:1439-1461; 1986. 208. Schultz, W.; Apicella, P.; Ljungbcrg, T., Responses of monkey dopamine neurons to reward and conditioned stimuli during succes- sive steps of learning a delayed response task. J. Neurosci. 13:900- 913; 1993. 209. Schwab, R. S. and Zieper, I. Effects of mood, motivation, stress and altermess on the pcrforrnancc in Parkinson's disease. Psychiat. NeurobioL, 150:345-357; 1965. 210. Schwarting, R.; Carey, R. J., Deficits in inhibitory avoidanc, after neurotoxic lesions of the ventral striatum are neurochemieally and behaviorally selective. Behav. Brain Res. 18:279-283; 1985. 211. Smith, G. P., Dopamine and food reward. Prog. Psyehobiol. Physiol. Psychol. 16:83-144; 1995. 212. Sorg, B. A.; Kalivas, P. W., Effects of cocaine and footshock stress on extracellular dopamine levels in the ventral striatum. Brain Res. 559:29-36; 1991. 213. Spivak, K. J.; Amit, Z., Effects of pimozide on appetitive behavior and locomotor activity: dissimilarity of effects when compared to extinction. Physiol. Behav. 36:457-463; 1986. 214. Staddon, J. E. R., Operant behavior as adaptation to constraint. J. Exp. Psyehol. Gen. 108:48-67; 1979. 215. Staddon, J. E. R. Adaptive behavior and learning. Cambridge, England: Cambridge University Press; 1983.; 216. Steele, M. A.; Weigl, P. D., Energeties and patch use in the fox squirrel Sciurus niger: responses to variation in prey profitability and patch density. Am. Midl. Nat. 128:156-167; 1992. 217. Stefurak, T. L.; van der Kooy, D., Tegmental pedunculopontine lesions in rats decrease saccharin's rewarding effects but no its memory-improving effects. Behav. Neurosei. 108:972-980; 1994. 218. Stellar, J. R.; Kelley, A. E.; Corbett, D., Effects of peripheral and central dopamine blockade on lateral hypothalamie self-stimulation: evidence for both reward and motor deficits. Pharmacol. Biocbem. Behav. 18:433-442; 1983. 219. Stdcker, E. M.; Zigmond, M. J. Recovery of function after damage.to central catecholamine-containing neurons: A neuroehemieal model for the lateral hypothalamie syndrome. In: Sprague, J. M., ed., Progress in Psychobiology and Physiological Psychology. New York: Academic Press;1976: 121-173. 220. Swanson, L. W.; Mogenson, G. J.; Gerfen, C. R.; Robinson, P., Evidence for a projection from the lateral preoptic area and substantia innominata to the "mesencephalic locomotor region". Brain Res. 295:161-178; 1984. 221. Swerdlow, N. R.; Braff, D. L.; Taaid, N.; Geyer, M. A., Assessing the validity of an animal model of deficient sensodmntor gating in schizophrenic patients. Arch. Gen. Psychiat. 51:139-154; 1994. 222. Szechtman, H.; Talangbayan, H.; Ganaran, G.; Dal, H.; Eilam, D., Dynamics of behavioral sensitization induced by the dopamine agonist quinpirole and a proposed central energy control mechanism. Psychopharmacot. 115:95-104; 1994. 223. Taylor, J. R.; Robbins, T. W., 6-Hydroxydopamine lesions of the nucleus accumbens but not the candate nucleus attenuate responding with reward-related stimului produced by intm-accumbens d- amphetamine. Psychopharmacol. 90:390-397; 1986. 224. Teuber, H.-L.; Proctor, F., Some effects of basal ganglia lesions in subhuman primates and man. Neuropsychol. 2:85-93; 1964. 225. Thorndike, E. L. Animal Intelligence. New York Macmillan; 1911. 226. Timberlake, W., Behavior systems and reinforcement: an integrative approach. J. Exp. Analy. Behav. 60:105-128; 1993. 227. Timberlake, W.; Allison, J., Response deprivation: an empirical approach to instrumental performance. Psyehol. Rev. 81:146-164; 1974. 228. Tombaugh, T. N.; Anisman, H.; Tombangh, J., Extinction and dopamine receptor blockade after intermittent reinforcement train- ing: failure to observe functional equivalence. Psychopharmacol. 70:19-28; 1980. 229. Tombangh, T. N.; Szostak, C.; Mills, P., Failure of pimozide to disrupt acquisition of light-dark and spatial discrimination problems. Psychnpharmaeoi. 79:161-168; 1983. 230. Tombaugh, T. N.; Szostnk, C.; Voomeveld, P.; Tombaugh, J. W., Failure to obtain functional equivalence between dopamine receptor I i blockade and extinction: evidence supporting an sensory-motor conditioning hypothesis. Pharmacol. Biochem. Behav. 16:67-72; 1982. 23 I. Treit, D.; Bea--ridge, K. C., A comparison of benzodiazepine, seroto- nin, and dopamine agents in taste-reactivity paradigm. Pharmaeol. Biocbem. Behav. 37:451-456; 1990. 232. Ungerstedt, U., Aphagia and adipsia after 6-hydroxydopamine induced degeneration of the nigro-striatal dopamine system. Acta Physiol Seand. 82 (suppl 367):95-122; 1971. 233. Ungerstedt, U.; Ljungberg, T., Central dopamine neurons and sen- sory processing. J. Psychiat. Res. 55:149-150; 1974. 234. Wadenberg, M. L.; Ericson, E.; Magnusson, O.; Ahlenins, S., Suppression of conditioned avoidance behavior by the local appli- cation of (-)-sulpiride into the ventral, but not the dorsal, striatum of I i the rat. Biol. Psychiat. 28:297-307; 1990. 235. Weiner, I.; Feldon, J., Facilitation of latent inhibition by haloperidol in rats. Psyehopharmaeol. 91:248-253; 1987. 236. Whishaw, I. Q.; Koroelsen, R. A., Two types of motivation revealed by ibotenie acid nucleus accumbens lesions: dissociation of food carrying and hoarding and the role of primary and incentive motivation. Bchav. Brain Res. 55:283-295; 1993. 237. Whishaw, I. Q.; O'Connor, W. T.; Dunnctt, S. B., The contributions of motor cortex, nigrostriatal dopamine and candate-putamcn to skilled forelimb use in the rat. Brain 109:805-843; 1986. 238. White,, I. M.; Ciancone, M. T.; Haraez, J. L.; Rebec, G. V., A lever- release version of the conditioned avoidance paradigm: Effects of haloperidol, clozapine, sulpiride and BMY-14802. Pharmacol. Bio- chem. Behav. 41:29-35; 1991. 239. White, N. M., Control of sensorimotor functions by dopaminergic nigrostriatal neurons: influence on eating and drinking. Neurosci. Biobehav. Rev. 10:15-36; 1986. 240. Wicscnfeld, Z.; Halpern, B. P.; Tapper, D. N., Licking behavior: evidence of a hypoglossal oscillator. Science 196:1122-1124; 1977. 241. Williams, B. A. Reinforcement, choice, and response strength. In Atkinson, R. C., Herrnstein, R. J., Lindzey, G., Lute, R. D. (eds.) Stevens' Handbook of Experimental Psychology, v.2; New York: John Wiley and Sons;1988:167-244. 242. Wiliner, P., Dopamine and depression: a review of recent evidence 11: theoretical approaches. Brain Res. 6:225-236; 1983. 243. WHiner, P. Depression: A Psychobiological Synthesis. New York: Wiley; 1985.; 244. Willner, P.; Chawala, K.; ~ampson, D.; Sophokleons, S.; Muscat, R., Tests of functional equivalence between pimozide pretreatment, extinction and free feeding. Psychopharmacol. 95:423-426; 1988. 245. Wilson, C.; Nomikos, G. G.; Collu, M.; Fibiger, H. C., Dopaminergic correlates of motivated behavior: importance of drive. J. Neurosci. 15:5169-5178; 1995. 246. Winn, P.; Robbins, T. W., Comparative effects of infusions of 6- hydroxydopamine into nucleus accumbcns and anterolateral hyl~)- thalamus induced by 6-hydroxydopamine on the response to dopa- mine agonists, body weight, locomotor activity and measures of exploration in the rat. Neuropharmacol. 24:25-31; 1985. 247. Wise, R. A., Catecholaminc theories of reward: a critical review. Brain Res. 152:215-247; 1978. 248. Wise, R. A., Neuroleptics and operant behavior: the anhedonia hypothesis. Behav. Brain Sci. 5:39-87; 1982. 249. Wise, R. A., The anhedonia hypothesis: mark IlL Behav. Brain Sci. 8:178-186; 1985. 250. Wise, R. A.; Bozarth, M. A., A psychomotor stimulant theory of addiction. Psychol. Rev. 94:469-492; 1987.

rg~,c , )pa - rv. ot' [" it-~,. BEHAVIORAL FUNCTIONS OF NUCLEUS ACCUMBENS DOPAMINE 359 251. Wise, R. A.; Colle, L. M., Pimozide attenuates free feeding: best scores analysis reveals a motivational deficit. Psychopbarmacol. 84:446-45l; 1984. 252. Wise, R. A.; Raptis, L., Effects of pre-feeding on food-approach latency and food consumption speed in food deprived rats. Physiol. Behav. 35:961-963; 1985. 253. Wise, R. A.; Spindler, J.; De Witt, H.; Gerber, G. J., Neuroleptic- induced "anhedonia" in rats: pimozide blocks reward quality of food. Science 201:262-264; 1978. 254. Wise, R. A.; Spindler, J.; Legult, L., Major attenuation of food reward with performance-sparing doses of pimozide in the rat. Can. J. Psychol. 32:77-85; 1978. 255. Wolterink, G.; Phillips, G.; Cador, M.; Donselaar-Woltednk, I.; Robbins, T. W.; Everitt, B. J., Relative roles of ventral striatal D1 and D2 dopamine receptors in responding with conditioned reinforcement. Psychopharmacol. 110:264-355; 1993. 256. Yang, C. R.; Mogenson, G. J., Electrophysiological responses of neurons in the nucleus accumbens to hippocampal stimulation and the attenuation of the excitatory responses by the mesolimbic dopaminergic system. Brain Res. 324:69-84; 1984. 257. Yang, C. R.; Mogenson, G. J., Hippocarnpal signal transmission to the pedunculopontine nucleus and its regulation by dopamine D-2 receptors in the nucleus accumbens: An electrophysiological and behavioral study. Neurosci. 23:1041-1055; 1987. 258. Yim, (2. Y.; Mogenson, G. J., Responses of nucleus accumbens neurons to amygdala stimulation and its modification by dopamine. Brain Res. 239:401-405; 1982. 259. Yoshida, M.; Yokoo, H.; Mizoguchi, K.; Kawahara, H.; Tsuda, A.; Nishikawa, T.; Tanaka, M., Eating and drinking cause increased dopamine release in the nucleus accubens and ventral tegmental area in the rat: Measurement by in vivo microdialysis. Neurosci. Lett. 139:73-76; 1992. 260. Young, A. M. J.; Joseph, M. H.; Gray, J. A., Latent inhibition of conditioned dopamine release in rat nucleus accumbens. Neurosci. 54:5-9; 1993. 261. Young, P. T. Motivation and Emotion. New York: John Wiley & Sons; 1961. 262. Zahm, D. S.; Brog, L S., On the significance ofsubterritories in the • 'accumbens" part of the rat ventral striatum. Neurosci. 50:751-767; 1992. 263. Zahm, D. S.; Heimer, L., Two transpallidal pathways originating in the rat nucleus accumbens. J. Comp. Neurol. 302:437-446; 1990. 264. Zeigler, H. P., Trigeminal deafferentation and hunger in the pigeon (Columba livia). J. Comp. Physiol. Psychol. 89:827-844; 1975. 265. Zeigler, H. P.; Jaequin, M. F.; Miller, M. G., Trigeminal orosensation and injestive behavior in the rat. Prog. PsychobioL Physiol. Psychol. 11:63-196; 1985. 266. Zeigler, H. P.; Miller, M. M.; Levine, R. R., Trigeminal nerve and eating in the pigeon (Columba livia): neurosensory control of the consummatory response. J. Comp. Physiol. Psychol. 89:845-858; 1975. 267. Zipf, G. K. Human Behavior and the Principle of Least Effort. Cambridge, MA: Addison-Wesley Press; 1949.