INTRODUCTION

Conditioned stimuli (CSs) previously paired with drugs of abuse can induce drug ‘craving’ (Ehrman et al, 1992) and relapse (Gawin and Kleber, 1986; Kosten et al, 2006). Indeed, an insidious feature of addiction is the persistent ability of drug-paired CSs to induce and maintain drug-seeking, even long after the last drug-taking experience (Gawin and Kleber, 1986; Robinson and Berridge, 1993). When rats are allowed to respond for presentations of a CS previously paired with heroin, cocaine or sucrose (Di Ciano and Everitt, 2004a), ethanol (Shahan, 2002), or water (Zimmerman, 1957), such responding can be persistent, even in the absence of the primary reinforcer. This property of the CS to act as a conditioned reinforcer is further exacerbated by treatment with psychomotor stimulant drugs and can result in very persistent responding that is resistant to extinction (Robbins, 1976, 1978). Persistent responding with conditioned reinforcement has been suggested to exhibit ‘functional autonomy’ (Zimmerman, 1957) in view of its continued performance in the absence of the primary goal (Di Ciano and Everitt, 2004a; Parkinson et al, 2005) and may reflect the automatic processes that influence drug addiction (Tiffany, 1990).

Several interconnected neural structures are implicated in control over drug-seeking by drug-paired CSs, including the basolateral amygdala (Whitelaw et al, 1996; Meil and See, 1997), the nucleus accumbens (NAcc) (Di Ciano and Everitt, 2001; Park et al, 2002; Ito et al, 2004), and the prefrontal cortex (Hutcheson and Everitt, 2003; Fuchs et al, 2004b; Kalivas et al, 2005). The dorsal striatum (DStr) has also been implicated in drug seeking and drug taking under the influence of CSs by both rats (Ito et al, 2002; Fuchs et al, 2006) and humans (Volkow et al, 2006). Given the postulated functions of the DStr in habit learning (Yin et al, 2004) and also in habitual drug seeking and drug taking (Everitt and Robbins, 2005), it was hypothesized that the DStr may be implicated in the persistent effects of conditioned reinforcement.

The purpose of the present study was thus to investigate the neural basis of the control of persistent responding for conditioned reinforcers, as assessed using a procedure in which the response producing the conditioned reinforcer is independent of the response that previously produced the drug (Robbins, 1976; Di Ciano and Everitt, 2004a), thus providing a stringent criterion for conditioned reinforcement (Mackintosh, 1974). Different subregions of the striatum were inactivated to analyze any differential involvement of the NAcc core and NAcc shell subregions, as well as the DStr in persistent responding with drug-associated conditioned reinforcement. We also investigated the effects of inactivation of the same striatal subregions before the CS (cue)-induced reinstatement of drug-seeking and the reacquisition of responding with conditioned reinforcement after extinction of the instrumental response, to determine whether any effects on conditioned reinforcement were independent of the original drug-reinforced contingencies or instrumental responding, respectively.

METHODS

Animals

Sixty-one (17 DStr, 26 NAcc core, and 18 NAcc shell) male Lister hooded rats weighing 280–300 g at the time of surgery (Charles River) were individually housed, and tested in operant chambers, as previously described (Di Ciano and Everitt, 2004a). Experiments were conducted in accordance with the United Kingdom 1986 Animals (Scientific Procedures) Act.

Surgery

Catheter

Catheter implantation and stereotaxic surgery were performed on the same occasion, with catheter implantation preceding stereotaxic procedures. Rats were anesthetized with ketamine hydrochloride (100 mg/kg intraperitoneal (i.p.), Ketaset, Dunlops, Dumfries, UK) and xylazine (9 mg/kg i.p., Rompun, Dunlops) and supplemented with ketamine as needed (∼20 mg). All rats were implanted with a single catheter in the right jugular aimed at the left vena cava. Catheters were purchased ready-made and consisted of a 22 g cannulae attached to Silastic tubing (0.012 inner diameter) and fixed to nylon mesh (Camcaths (www.camcaths.com), Cambridge, UK). The mesh end of the catheter was sutured subcutaneously (s.c.) on the dorsum. All surgical instruments were thoroughly sterilized before surgery. To prevent infection, rats were treated postsurgically with 10 mg/kg Baytril (Dunlops) s.c. for 8 days.

Stereotaxic

Immediately following catheter implantation, rats were positioned in a stereotaxic frame for insertion of guide cannulae. Guide cannulae consisted of 24 g thin-walled stainless steel tubing (Cooper's Needleworks, Harlow, UK) and lowered to 2.5 mm above the target site for the NAcc core and NAcc shell and 1.0 mm above the target site for the DStr (core: +1.2 AP, +1.8 ML, −4.7 DV; shell: +1.0 AP, +1.0 ML, −5.3 DV; DStr: −1.2 AP, ±3.0 ML, −5.0 DV; incisor bar −3.3 mm (Paxinos and Watson, 1986)). Cannulae were secured with stainless steel screws and dental acrylic; 29 g wire stylets (Cooper's Needleworks) were inserted into the length of the guide cannulae to maintain patency.

Procedure

Figure 1 illustrates the sequence of behavioral testing, summarized below.

Figure 1
figure 1

Schematic of time course of behavioral testing, the behavioral measure under study, and the time of microinfusions. Dark lines represent the response that was reinforced, solid lines indicate that the response was recorded but had no programmed consequences, and stippled lines represent times at which the levers were retracted. The type of reinforcer earned is indicated above the dark lines (CS, conditioned reinforcer). Vertical arrows illustrate the times at which animals received infusions of either vehicle (gray arrows) or inactivation (dark arrows) in a between-subjects or within-subjects (open double arrow) counterbalanced order.

Full size image

Acquisition of a nose-poke response for i.v. cocaine and drug-paired stimulus under continuous reinforcement

Daily experimental testing began 7–10 days after surgical procedures. On each testing day, rats were connected to the i.v. line before the start of the training session. During 8 days of training, rats received a single intravenous (i.v.) cocaine infusion (0.25 mg/infusion/0.1 ml/5 s) following every nose poke into the magazine (fixed ratio 1, time-out 20 s). Each cocaine infusion was accompanied with illumination of a stimulus light above one of the retracted levers for 20 s. During this 20 s ‘time-out’, the house light was extinguished and nose pokes were recorded but had no consequences. Following this 20 s stimulus, the house light was again illuminated and the stimulus light was extinguished. The side of the stimulus light was counterbalanced between left and right sides for individual animals. ‘Priming’ injections of cocaine were never given. To prevent accidental overdose, rats were limited to 30 infusions in a 60 min session.

Acquisition of a novel lever press response for a drug-paired conditioned reinforcer

Following nose poke training with cocaine and associated CS reinforcement, rats were tested for acquisition of a new lever press response with the CS alone as the reinforcer, in the absence of any further pairings with the drug. Rats were placed in the operant chambers (not attached to the i.v. lines), with the pumps turned off to prevent the sound of the motor. Two novel levers were inserted into the operant chamber (for the first time) and presses on the lever opposite the CS were reinforced with a 1 s presentation of this stimulus light, and a 1 s extinction of the house light. Responses were reinforced under a variable ratio 1 : 3 schedule of reinforcement. Responses on the inactive lever were recorded but had no programmed consequences. These sessions were 30 min in length.

Persistence and reacquisition of responding for a drug-paired conditioned reinforcer

Animals were tested in daily sessions 5–7 days a week for approximately 40 sessions. After approximately 20 (±5) days, the CS was removed for 6 days to extinguish the lever press. All other parameters were as above except that lever presses had no programmed consequences. After 6 days of extinction, the conditioned reinforcer was again made contingent upon lever presses for at least 2 days. During these reacquisition sessions, presses on the active lever resulted in presentations of the conditioned reinforcer, as described above. Rats were assigned to vehicle or inactivation groups at reacquisition such that their baseline rates of responding before extinction were similar.

CS (cue)-induced reinstatement

To measure CS (cue)-induced reinstatement of responding, the conditioned reinforcer was again made contingent on nose poking. To determine the effect of reintroduction of the conditioned reinforcer on nose pokes, responding was compared to a ‘no CS’ session in which the conditioned reinforcer was not available (ie, nose pokes had no programmed consequences). During these 60 min sessions, both levers were retracted. Reinstatement sessions and ‘no CS’ nose poke sessions were identical to self-administration sessions, but for turning off the infusion pump and not connecting the rats to i.v. lines’ preventing cocaine infusions. During the ‘no CS’ sessions, the contingent CS was also turned off. At least 2 days of ‘no CS’ sessions separated the two reinstatement sessions.

Microinfusions

Intracerebral microinfusions were identical to published procedures (McFarland and Kalivas, 2001) and consisted of 0.3 μl infused over 54 s simultaneously to both sides of the brain, followed by a 60 s postinfusion diffusion time. The GABAB receptor agonist baclofen and the GABAA agonist muscimol were delivered in a mixture at the doses of 0.1 nmol (baclofen) and 0.01 nmol (muscimol). All rats received one or two ‘sham’ infusions of phosphate buffer (PB), on separate days and at least 4 days before inactivations, to habituate the animal to the injection procedure. All infusions were delivered 5 min before rats were placed in operant chambers for testing and were separated by at least 1 day of baseline responding for the conditioned reinforcer.

Figure 1 illustrates the sequence of training and microinfusions. The NAcc core, NAcc shell, or DStr were inactivated before three sessions, to measure the effects on (1) persistence of responding for the drug-paired conditioned reinforcer: in a counterbalanced order, rats received either inactivation or vehicle infusion. These infusions were given before test sessions after rats had been working for the conditioned reinforcer for approximately 14–20 days. A minimum of 14 days was selected because it has previously been demonstrated that at this time responding for the CS was independent of a representation of the drug with which it had previously been paired (Di Ciano and Everitt, 2004a); (2) reacquisition of lever pressing for the drug-paired conditioned reinforcer: in a between-subjects design, rats then received either vehicle or inactivating infusions before the first session after extinction in which the conditioned reinforcer was again available contingently; and (3) CS (cue)-induced reinstatement of the nose poke response: in a counterbalanced order, rats received vehicle and inactivation infusions as described above before CS (cue)-induced reinstatement of nose pokes. This session was identical to that described above for self-administration training except that rats did not receive cocaine infusions. During these 60 min sessions, levers were retracted.

Histology

At the end of testing, rats were anesthetized with an overdose of sodium pentobarbitone (1.5 ml per animal i.p., Dolethal, Dunlops) and perfused transcardially with isotonic saline, followed by 4% paraformaldehyde in 0.2 M PB. Brains were then removed and postfixed before being transferred to a 20% sucrose solution in 0.01 M phosphate-buffered saline for about 24 h before being sectioned at 60 μm using a freezing microtome. Every third section was mounted, stained with Cresyl Violet, and placements verified under a light microscope.

Statistical Analyses

For all sessions, lever presses or nose pokes were recorded. Data are presented as mean±SEM number of responses. For all comparisons, data were analyzed with repeated-measures ANOVAs with Group as the between-subjects factor. Lever presses and nose pokes were analyzed separately. Significant effects were followed by planned comparisons. For all analyses, a criterion of p⩽0.05 was selected for significance.

Acquisition of nose pokes for cocaine and associated conditioned reinforcer

Acquisition of nose pokes for self-administered cocaine was analyzed with a three-way Group (3 levels) × Day (8 levels) × behavioral measure (2 levels) ANOVA with Group as the between-subjects variable.

Acquisition and persistence of responding for the drug-paired conditioned reinforcer

For acquisition of lever press responses for the CS, data were analyzed with a repeated-measures Group (3 levels) × Day (13 levels) × Lever (2 levels) ANOVA with Group as the between-subjects variable. Significant interactions were further analyzed with two-way, one-way ANOVAs, and planned comparisons.

Effect of inactivations of NAcc core, NAcc shell or DStr on persistence of responding for the drug-paired conditioned reinforcer

Data were analyzed with repeated-measured Lever (2 levels) × Treatment (2 levels; vehicle or inactivation) ANOVAs. Significant interactions were followed by one-way ANOVAs and planned comparisons. Nose pokes were analyzed separately as an independent measure to control for any effects of treatments to produce motor activation.

Reacquisition of responding for the contingent drug-paired conditioned reinforcer

To demonstrate the reacquisition of responding for the drug-paired conditioned reinforcer, data from all the rats that received vehicle on the reacquisition day 1 were collapsed across groups for the baseline day (day before extinction), extinction day 6, reacquisition day 1, and reacquisition day 2. Data were collapsed across groups, as only a subset of rats received testing on reacquisition day 2 and analysis by brain area was not possible due to the small Ns in the NAcc shell group (vehicle, n=3; inactivation, n=4). Data were presented in three ways: (1) number of responses; (2) proportion of total responses that were on the active lever. This was to determine whether changes in responding were aimed at maintaining a constant level of conditioned reinforcer presentations. Proportion of active lever responses was computed by dividing the number of active lever presses by the sum of active lever presses, inactive lever presses, and nose pokes; and (3) percent change in responding from extinction day 6. This was computed to control for baseline levels of nonspecific motor effects and to provide a better measure of response discrimination during relearning. Our pilot studies revealed that when compared to extinction day 6, the first day of reacquisition of responding was characterized by a decrease in responses on the inactive lever and nose pokes, while the active lever responses remained the same.

To test the effect of omission and reintroduction of the contingent conditioned reinforcer on behavior, the number of responses made was analyzed with a repeated-measures Day (4 levels) ANOVA separately for each behavioral measure. Further analyses with planned comparisons were conducted between the baseline day and each of the other days. The proportion of all responses that were made on the active lever was analyzed with planned comparisons between baseline and every other day. Percent change from extinction day 6 was analyzed with three separate repeated-measures Day (4 levels) × Measure (2 levels) ANOVAs, one comparing the active and inactive lever, one comparing the active lever and nose pokes and the other to compare the inactive lever to nose pokes.

To test whether reacquisition on the first day may reflect in part the ability to discriminate between the various response options as the animal reacquired the correct response for the conditioned reinforcer, the number of responses and percent change from extinction day 6 were further analyzed with planned comparisons between extinction day 6 and reacquisition day 1, separately for each behavioral measure.

Effect of inactivation of the DStr, NAcc core, or NAcc shell on reacquisition

The number of responses on extinction day 6 was compared to reacquisition day 1 with a repeated-measures Group (2 levels; vehicle or inactivation) × Day (2 levels; extinction day 6 or reacquisition day 1) × Measure (2 levels; active lever, inactive lever) ANOVA, with Group as the between-subjects variable. Nose pokes were analyzed separately as an independent behavioral measure to control for treatment effects on nonspecific motor activation. The proportion of total responses that were on the active lever was analyzed with a one-way ANOVA on the effect of Day (2 levels) separately for each treatment group. To determine the effects of inactivation of either the NAcc core, NAcc shell, or DStr on percent change from baseline, data were analyzed with a repeated-measures Lever (2 levels) × Treatment (2 levels) ANOVA with Treatment as the between-subjects factor. Significant interactions were followed with one-way ANOVAs or planned comparisons. Nose pokes were analyzed separately as an independent instrumental measure to control for treatment effects on nonspecific motor activation.

CS (cue)-induced reinstatement of nose pokes for the drug-paired conditioned reinforcer

Two planned comparisons were conducted separately for each group: (1) comparing the ‘no CS’ baseline day to CS (cue)-induced reinstatement following vehicle infusion, and (2) comparing CS (cue)-induced reinstatement following vehicle infusion to CS (cue)-induced reinstatement following inactivation.

RESULTS

Histology

Figure 2 illustrates the location of tips of guide cannulae in the NAcc core (circles, n=26), NAcc shell (triangles, n=18), or DStr (squares, n=17). Coronal sections were taken from Paxinos and Watson (1986).

Figure 2
figure 2

The location of tips of guide cannulae in the NAcc core (circles, n=26), NAcc shell (triangles, n=18), or DStr (squares, n=17). Coronal sections were taken from Paxinos and Watson (1986).

Full size image

Acquisition of a Nose Poke Response for Cocaine and Drug-Paired CS

All groups of rats acquired a nose poke response for cocaine on the first day, and the number of infusions and number of responses remained stable across all 8 days of self-administration (Figure 3). A repeated-measures Day × Group × Measure ANOVA revealed a significant effect of behavioral measure only (F(1, 58)=57.28, p<0.001), suggesting that all groups of rats (DStr, n=17; core, n=26; shell, n=18) made more nose pokes than the number of cocaine infusions earned and that the number of responses or reinforcers remained stable across days.

Figure 3
figure 3

Mean±SEM number of nose pokes made (filled symbols) and cocaine infusions earned (open symbols) during acquisition of cocaine self-administration with associated light conditioned stimulus during 8 days of training. Analysis revealed no differences between the three groups (squares: dorsal striatum (DStr), n=17; circles: core, n=26; triangles: shell, n=18). The number of responses made was higher than the number of infusions earned (p<0.05).

Full size image

Acquisition and Persistence of Responding for a Drug-Paired Conditioned Reinforcer

Rats acquired the lever press response on the active lever for the drug-paired conditioned reinforcer (Figure 4, upper panel). Responding on the active lever reached maximal levels by day 5, after which responding remained stable for the duration of testing; responding on the inactive lever remained stable after the second day and was significantly and consistently lower than responding on the active lever. A Day × Lever × Group ANOVA revealed a significant Lever × Day interaction (F(12, 624)=7.72, p<0.001), indicating that responding on the two levers varied differently across days. Follow-up analysis with ANOVAs on the effect of Day for each lever revealed a significant effect of Day for both the active (F(12, 624)=7.88, p<0.001) and the inactive levers (F(12, 624)=1.84, p=0.04), suggesting that the significant interaction was due to a greater increase in responding over days on the active lever as compared to the inactive lever (DStr, n=12; core, n=26; shell, n=18). Five rats from the DStr group were excluded from analyses because of treatments administered on the first day of acquisition (pilots).

Figure 4
figure 4

Mean±SEM number of lever presses (top panel) on the active (filled symbols) and inactive (open symbols) levers, and nose pokes (bottom panel) made, during the first 13 days of acquisition of responding for a cocaine-paired conditioned reinforcer. For the lever press data, a significant Lever × Day interaction was revealed (p<0.05) with no group effects. For the nose-poke data, a significant effect of Day was revealed (p<0.05) (dorsal striatum (DStr), n=12; NAcc core, n=26; NAcc shell, n=18).

Full size image

During 13 days of acquisition of the lever press response for the drug-paired conditioned reinforcer, nose pokes (Figure 4, bottom panel) decreased over days. ANOVA revealed a significant Day × Group interaction (F(24, 636)=1.92, p=0.005), which follow-up analysis with one-way ANOVAs on the effect of Day for each group revealed was due to a more significant decrease in the NAcc core (F(12, 300)=7.199, p<0.001; n=26) and NAcc shell (F(12, 204)=4.672, p<0.001; n=18) groups, than the DStr group, for which the effect of Day approached significance (F(12, 132)=1.756, p=0.062; n=12).

Effect of Inactivation of the DStr, NAcc Core, or NAcc Shell on the Persistence of Responding for the Drug-Paired Conditioned Reinforcer

Table 1 provides a summary of the results presented in Figures 5, 7, and 8.

Table 1 Summary of the Findings from the Present Study of Effects of Inactivations of the DStr, NAcc core, or NAcc Shell on Persistent Responding for a Drug-Paired Conditioned Reinforcer, Its Reacquisition after Extinction When the Contingent Drug-Paired Conditioned Reinforcer was Omitted, or CS (cue)-Induced Reinstatement of the Nose-Poke Response that had Previously Resulted in Self-administered Cocaine and the Associated Stimulus During Training
Full size table
Figure 5
figure 5

Effect of inactivation of the dorsal striatum (DStr, left panel; n=14), nucleus accumbens (NAcc) core (middle panel; n=24), and NAcc shell (right panel; n=16) on persistent lever presses for the drug-paired conditioned reinforcer. Inactivation of either the DStr or NAcc core decreased responding on the active lever only. *Significant effect of inactivation compared to vehicle on the active lever for that Group.

Full size image

As illustrated in Figure 5, inactivation of either the DStr (n=14; left panel) or NAcc core (n=24; middle panel), but not the NAcc shell (n=16; right panel), decreased persistent responding for the cocaine-paired conditioned reinforcer, while inactive lever presses and nose pokes were not affected. ANOVA revealed a significant Lever × Treatment interaction for both the DStr (F(1, 13)=7.06, p=0.02) and NAcc core (F(1, 23)=8.073, p<0.01), suggesting that inactivation had differential effects on presses on the active or inactive levers. Follow-up analysis revealed that this interaction was due to an effect on the active lever only for both the DStr (F(1, 13)=5.36, p=0.04) and NAcc core (F(1, 23)=5.19, p=0.03) groups, suggesting that inactivation of the DStr or NAcc core selectively decreased responding for the conditioned reinforcer on the active lever only. For the NAcc shell, ANOVA revealed an effect of Lever only (F(1, 15)=52.28, p<0.001), indicating that responding on the active lever was higher than on the inactive lever for both groups. Two outliers were removed from each group and one rat in the DStr group died before the test for persistence.

Reacquisition of Responding for the Drug-Paired Conditioned Reinforcer

Reacquisition

As illustrated in Figure 6, the number of active lever responses (top panel, line graph) was decreased by day 6 of extinction and returned to baseline levels by the second day of reacquisition; the proportion of total responses that were on the active lever remained constant on all days in which the conditioned reinforcer was available (top panel; histogram); responses on the inactive lever or nose pokes varied to a lesser degree, and most notably they decreased on reacquisition day 1. When expressed as percent change from baseline (bottom panel), the changes in responding on the active lever were more apparent than during responding on either the inactive lever or for nose pokes. Analysis of the number of responses (top panel) revealed a significant effect of Day only for the active lever (F(3, 36)=12.483, p<0.001), suggesting that active lever presses varied depending on the presence of the conditioned reinforcer. Planned comparisons revealed that for the active lever and nose pokes responding on the first day of reacquisition was significantly lower than baseline (active lever: F(1, 12)=16.97, p=001; nose pokes: F(1, 12)=7.358, p=0.019), while for the inactive lever this difference approached significance (F(1, 12)=3.734, p=0.077). Responding on the active lever was also significantly lower than baseline on extinction day 6 (F(1, 12)=33.752, p<0.001). The proportion of total responses that were on the active lever was constant during all the sessions in which the conditioned reinforcer was present, as planned comparisons revealed a significant difference between baseline and extinction day 6 only (F(1, 12)=67.042, p<0.001). When expressed as percent change from baseline (Figure 6, bottom panel), responding on the active lever was significantly higher and more variable than either presses on the inactive lever or nose pokes, as revealed by a significant Day × Measure interaction on the comparisons between the active and inactive levers (F(3, 36)=5.846, p=0.002) or active lever and nose pokes (F(3, 36)=4.725, p=0.007). Comparison of the inactive lever responses to nose pokes revealed only a significant effect of Day (F(3, 26)=4.078, p=0.014), confirming that behavior varied across days, but not differently between these two measures.

Figure 6
figure 6

Reacquisition of responding for the contingent drug-paired conditioned reinforcer after extinction by omission of the contingent conditioned reinforcer (CS) for 6 days. Data for the active lever (squares), inactive lever (triangles), or nose pokes (circles) are expressed as mean±SEM number of responses (top panel, line graph), the proportion of total responses that were on the active lever (top panel, histogram, right axis), or the percent change in responding from extinction day 6 (bottom panel) on the last day of responding for the conditioned reinforcer (base), the last day of extinction without the conditioned reinforcer (ext6), reacquisition day 1 (Re-Acq), and day 2 (Re-Acq2) for the CS. Dark lines represent significant main effects of day when compared across all 4 days (*p<0.05) or between extinction day 6 and reacquisition day 1 (+p<0.05). Filled symbols represent a significant difference from baseline for that group (p<0.05). **Represents a significant Day × Measure interaction between the active and inactive lever presses and active lever and nose pokes.

Full size image

On the first day of reacquisition, when expressed as either the number of responses (Figure 6, top panel, line graph) or percent change from extinction day 6 (Figure 6, bottom panel), responses on the inactive lever and nose pokes were decreased as compared to extinction day 6, while responses on the active lever were not different; this difference between the various measures was more apparent when expressed as percent change. ANOVA revealed a significant effect of Day for the inactive lever (responses: F(1, 12=9.922, p<0.01; percent: F(1, 12)=33.715, p<0.001) and nose pokes (response: F(1, 12)=10.165, p<0.01; percent: F(1, 12)=13.814, p=0.003) only, when extinction day 6 was compared to the first day of reacquisition, revealing that inactive lever presses and nose pokes decreased on the first day of reacquisition, while active lever responses remained the same, as compared to extinction day 6.

Effect of Inactivation of the DStr, NAcc Core, or NAcc Shell on the Reacquisition of Responding for the Drug-Paired Conditioned Reinforcer

Table 1 provides a summary of the results presented in Figures 5, 7, and 8.

Figure 7
figure 7

Effect of inactivation of the dorsal striatum (DStr, left panels), nucleus accumbens (NAcc core, middle panels), or NAcc shell (right panels) on the reacquisition of responding for the conditioned reinforcer after extinction by omission of the conditioned reinforcer for 6 days. Top panels: Mean±SEM number of responses on extinction day 6 (ext6) and the first day of reacquisition (Re-A) for the vehicle (veh) or inactivation (inact) groups. Histograms represent the proportion of the total responses that were on the active lever. **Represents a significant difference on reacquisition day 1 for that treatment group. Data for active lever presses (filled squares), inactive lever presses (open squares), or nose pokes (circles) are presented and significant main effects of Day and Day × Measure (inactive or active lever) interactions are represented by dark line and stippled lines, respectively. Bottom panels: percent change from extinction day 6 during the first day of reacquisition. Data for the active lever (dark symbols), inactive lever (open symbols), or nose pokes (gray symbols) are presented and significant main effects (*) or interactions (+) are indicated (p<0.05).

Full size image
Figure 8
figure 8

Effect of inactivation of the dorsal striatum (DStr, left panel; n=10), nucleus accumbens (NAcc) core (middle panel; n=8), or NAcc shell (right panel; n=6) on CS (cue)-induced reinstatement of nose pokes. Data shown are for the number of nose pokes made in the absence of the drug-paired conditioned reinforcer (gray bars) or during CS (cue)-induced reinstatement following either vehicle (open bars) or inactivation (dark bars) treatments. Inactivation of the NAcc core and NAcc shell decreased and increased CS (cue)-induced reinstatement, respectively, while inactivation of the DStr had no effect. *Significant difference compared to CS (cue)-induced reinstatement following vehicle infusion (p<0.05, planned comparisons).

Full size image

As illustrated in Figure 7, inactivation of either the DStr (left panels), NAcc core (middle panels), or NAcc shell (right panels) had differential effects on the reacquisition of responding for the drug-paired conditioned reinforcer after extinction by omission of the contingent drug-paired conditioned reinforcer.

DStr

Analysis of the number of responses made by the DStr group (Figure 7, top left panel) revealed a just significant Measure × Day × Group interaction (F(1, 12)=4.669, p=0.052) which follow-up analysis revealed was due to a Measure × Day interaction that just approached significance for the vehicle group (n=7; F(1, 6)=5.412, p=0.059), and a significant effect of Measure (F(1, 6)=9.375, p=0.022) and Day (F(1, 6)=6.765, p=0.041) for the inactivation group (n=7; one outlier removed), suggesting that the significant interaction was due to differential effects in the two groups. That is, for the inactivation group, presses on both the active and inactive levers decreased over days, while for the vehicle group the direction of change in responding on these levers was divergent, with active lever presses increasing, and inactive lever presses decreasing, on the first day of reacquisition. An effect of Day only was found for analysis of nose pokes (F(1, 12)=9.716, p=0.009), suggesting that they decreased in both groups. For the proportion of all responses that were on the active lever, the effect of Day just reached significance for the vehicle group (F(1, 6)=5.745, p=0.054), suggesting that responding only by the vehicle group was aimed at maintaining a constant level of conditioned reinforcer presentations.

Further analysis of effects of inactivation of the DStr with percent change from extinction day 6 (Figure 7, bottom left panel) revealed a significant Lever × Group interaction (F(1, 13)=4.62, p=0.05), which follow-up analyses revealed were due to a significant effect of Lever for the vehicle group (n=7) only (F(1, 6)=8.89, p<0.05), but not for the inactivation group (n=8, p>0.1). This suggests that responding by the vehicle group was greater on the active lever than on the inactive lever during reacquisition of the response, but not for the inactivation group. When expressed as percent change from baseline, no effects of treatments on nose pokes were found during reacquisition. Two rats were removed from analysis as it died before reacquisition testing.

NAcc core

Analysis of the number of responses made by the NAcc core groups (Figure 7, top middle panel; vehicle, n=12; inactivation, n=13) revealed a Measure × Day interaction approached significance (F(1, 23)=4.056, p=0.056), suggesting that the behavioral measures varied across days, but were the same for both groups. Follow-up analysis revealed a significant effect of Day (F(1, 23)=10.676, p<0.003) only for the inactive lever and nose pokes (F(1, 23)=10.171, p=0.004), suggesting that only inactive lever presses and nose pokes varied on the first day of reacquisition, by decreasing. For the proportion of total responses that were on the active lever, the effect of Day was significant for the vehicle group only (F(1, 11)=29.680, p<0.001), suggesting that responding only by the vehicle group resulted in a constant level of conditioned reinforcer presentations.

Further analysis of reacquisition data with percent change from extinction (Figure 7, bottom middle panel) revealed a significant effect of Lever (F(1, 22)=10.43, p=0.01) and Group (F(1, 22)=5.97, p=0.02), but no interaction (p>0.2) suggesting that responding by the inactivation group (n=12; one outlier removed) was decreased on both levers and was lower on both days than the vehicle group (n=12). No effects of treatments on nose pokes were found. One rat was removed from analysis as it died before reacquisition testing.

NAcc shell

Analysis of the number of responses made by the NAcc shell group (Figure 7, top right panel) during reacquisition revealed a significant effect of Measure only (F(1, 13)=22.688, p<0.001), suggesting that presses on the active lever were higher than those on the inactive lever for both the vehicle (n=6; one outlier removed) and inactivation (n=9) groups. For proportion of active lever presses, no significant effects were found.

Further analysis of the reacquisition data with percent change from extinction day 6 (Figure 7, bottom right panel) revealed no significant effects (vehicle, n=7; inactivation, n=8; one outlier removed). Three rats were removed from analysis as they died before reacquisition testing.

Effect of Inactivation of the DStr, NAcc core, or NAcc shell on CS (cue)-Induced Reinstatement

Table 1 provides a summary of the results presented in Figures 5, 7, and 8.

As illustrated in Figure 8, inactivation of the DStr, NAcc core, or NAcc shell had different effects on CS (cue)-induced reinstatement; inactivation of the DStr (n=10; left panel) had no effect, while inactivation of the NAcc core (n=8; middle panel) decreased, and inactivation of the NAcc shell (n=6; right panel) increased, CS (cue)-induced reinstatement. For all brain areas, planned comparisons revealed a significant difference between responding without the CS and CS (cue)-induced reinstatement following vehicle infusion (p<0.05). In addition, for the NAcc core and NAcc shell, planned comparisons revealed a significant difference between the effects of vehicle or inactivation treatment on CS (cue)-induced reinstatement (p<0.05), indicating that inactivation of the NAcc core or NAcc shell increased or decreased CS (cue)-induced reinstatement, respectively. Only a subset of rats were tested on CS (cue)-induced reinstatement and all data is presented.

DISCUSSION

The purpose of the present study was to investigate differential effects of inactivation of the NAcc core and NAcc shell subregions, as well as the DStr, on persistent responding with drug-paired conditioned reinforcers and on responding during CS (cue)-induced reinstatement. Inactivation of these structures using infusions of baclofen and muscimol (McFarland and Kalivas, 2001) produced different patterns of effects (Table 1). DStr inactivation blocked persistent responding with conditioned reinforcement, and its reacquisition after extinction, when that responding had been acquired for the drug-paired conditioned reinforcer alone. By contrast, such inactivation had no effect on CS (cue)-induced reinstatement, when the same CS was presented contingently for nose poke responding that had originally resulted in i.v. drug delivery. In comparison, inactivation of the NAcc shell region had no effect on persistent responding with conditioned reinforcement, and actually increased responding during CS (cue)-induced reinstatement. Inactivation of the NAcc core not only blocked CS (cue)-induced reinstatement, but also resulted in a decrease in persistent responding and reacquisition after extinction. These results indicate that specific regions of the striatum are uniquely implicated in the control of responding by drug-paired conditioned reinforcers according to whether the responding was previously associated with the drug reinforcer.

Different Measures of Persistent Responding for a Drug-Paired Conditioned Reinforcer Measure Different Aspects of Drug Addiction

Stimuli associated with drug reinforcers can affect behavior in several ways. For example, they can act as conditioned reinforcers that maintain responding under second-order schedules of drug reinforcement (when responding is maintained by a combination of the drug and drug-paired conditioned reinforcers (Arroyo et al, 1998)) and can also reinstate responding when reintroduced after extinction of responding (de Wit and Stewart, 1981; Meil and See, 1996). In addition, as shown in this study they can also act, apparently indefinitely, as reinforcers in their own right (Di Ciano and Everitt, 2004a). This persistence has been demonstrated previously for stimuli associated with heroin (Di Ciano and Everitt, 2004a), sucrose (Beninger et al, 1980), ethanol (Shahan, 2002), water (Zimmerman, 1957; Robbins, 1976), or brain stimulation reward (Robbins and Koob, 1978), suggesting that conditioned reinforcement may be important in many forms of compulsive behavior. The nature of the response changes over time (Di Ciano and Everitt, 2004a), such that during early acquisition responding for the conditioned reinforcer is sensitive to devaluation of its association with cocaine, while after repeated responding, the response persists even despite such devaluation. An insensitivity to the value of the reward previously paired with conditioned reinforcers is believed to be a feature of addiction, as, for example, drug addicts show an insensitivity to the reward value of monetary outcomes (conditioned reinforcers), rating $1000 the same as $10 (Goldstein et al, 2007).

By comparison to persistent responding, reacquisition of responding may reflect a number of variables important for relearning the instrumental response for the conditioned reinforcer after instrumental extinction. First, animals must relearn the association between the response and the outcome. Indeed, removal of the contingent drug-paired conditioned reinforcer decreased responding on the active lever only, suggesting that this response was maintained by contingent presentation of the conditioned reinforcer. The subsequent reacquisition curve was more rapid than initial learning, typical of the savings in relearning of an instrumental response after extinction. Second, the predictive significance of the context must be reestablished, as during extinction the ability of the context to activate behavior (Holland, 1977; Martin-Iverson et al, 1986; Wise and Bozarth, 1987) is decreased. The opposite is also true, as non-contingent presentations of a reward can produce a context-mediated increase in the persistence of conditioned reinforcement (Beninger et al, 1980; Shahan and Burke, 2004), providing support for the suggestion that the context mediates responding for the conditioned reinforcer (Rescorla and Solomon, 1967). Third, animals must also relearn to discriminate the various response options. Thus, the altered behavioral repertoire observed on reacquisition day 1, with a selective change (decrease) in nose pokes and inactive lever presses, may represent a compensatory shift in behavior to maintain conditioned reinforcer presentations in the face of an overall decrease in behavior. One notable difference between the procedure used here and the typical model of CS (cue)-induced reinstatement (See, 2005) is that in the present case, the response for the conditioned reinforcer never previously resulted in drug, and thus, conditioned reinforcement is measured free of any association of the response for the drug (Mackintosh, 1974).

Persistent Responding for a Conditioned Reinforcer

This study has shown that different sectors of the striatum participate differentially in some of these functions. Thus, persistent responding with drug-paired conditioned reinforcers depended on the NAcc core and the DStr, both of which are also important in the control of responding under a second-order schedule of cocaine reinforcement (Ito et al, 2000, 2004; Di Ciano and Everitt, 2004b; Vanderschuren et al, 2005), suggesting that both are involved in conditioned reinforcement processes. The fact that DStr inactivation did not affect CS (cue)-induced reinstatement suggests that the acquisition of a new response procedure and CS (cue)-induced reinstatement taps different aspects of control over behavior by conditioned reinforcers. That the DStr inactivation affected only the former whereas the NAcc core lesion blocked both suggests that the DStr is more selectively involved in persistent responding with drug-paired conditioned reinforcers (and thus the reacquisition of this control after extinction). The contrasting effects of DStr inactivation on CS (cue)-induced reinstatement and persistent responding following acquisition of a novel response for the conditioned reinforcer imply that other factors may control CS (cue)-induced reinstatement. Whereas the new response with conditioned reinforcement has never been paired with the drug itself this is not the case in CS (cue)-induced reinstatement procedures. Thus, CS (cue)-induced reinstatement may depend in part on the retrieval of an association with the drug and the response delivering it.

Consistent with the suggestion that the DStr is involved in the mediation of conditioned reinforcement, inactivation of the DStr affected reacquisition assessed by all measures, while NAcc core inactivation only affected reacquisition when expressed as percent change from baseline, or the proportion of total responses that were made on the active lever. This suggests that the DStr is involved more specifically with responding for the conditioned reinforcer, while the NAcc core may mediate some of the activational or discriminative aspects of behavioral control during acquisition of an instrumental response. Indeed, blockade of glutamate receptors in the NAcc core attenuated the acquisition of instrumental responding (Kelley et al, 1997), and discriminative control over responding (Hauber et al, 2000; Di Ciano et al, 2001; Di Ciano and Everitt, 2001). In the present study, inactivation of the NAcc core decreased discriminated responding, as changes in lever presses were the same for presses on both the active and inactive levers. Further, the lack of effect on nose pokes suggests that, in the present study, inactivation of the NAcc core did not nonspecifically affect motor behavior, but instead, the vigor of the responding that was appropriate for the outcome (ie, the lever press). Indeed, the contextual control over responding (Gratton and Wise, 1994; Di Ciano et al, 1998; Sellings and Clarke, 2003), and response vigor per se (Salamone et al, 1994) have both been shown to be mediated by dopamine (DA) receptors in the NAcc core. Together, this suggests that the NAcc core may provide discriminative control over behavior via glutamate receptors that occurs on a baseline of incentive motivation induced by the context and that is mediated by DA receptors in the NAcc core.

This account assumes that persistent responding maintained by the conditioned reinforcer following acquisition of a new response does not simply reflect a ‘drug-seeking’ response, but, rather, may reflect an independent mechanism of conditioned reinforcement. This is supported by evidence mentioned in the ‘Introduction’, that food-paired conditioned reinforcers appear impervious to food devaluation induced by satiety (Parkinson et al, 2005), while extinction of a CS-cocaine association did not decrease conditioned reinforcement for a CS previously paired with cocaine (Di Ciano and Everitt, 2004a). The ‘value’ of the conditioned reinforcer may be necessary for the acquisition of the response, but less important for its maintenance, as the independence of the CS from the drug develops over time and/or experience (Di Ciano and Everitt, 2004a). It is thus plausible that the conditioned reinforcer maintains responding indefinitely because such responding becomes somewhat habitual in nature. It appears likely that some of these mechanisms contribute to those aspects of second-order schedule performance that depends on the DStr (Ito et al, 2002; Vanderschuren et al, 2005). Recent analyses suggest that the dorsomedial and dorsolateral sectors of the DStr subserve different functions in instrumental learning, the latter being especially important in S-R habit learning (Yin and Knowlton, 2006). It thus seems plausible that persistent responding with conditioned reinforcers acquires habit-like properties and the findings are consistent with the hypothesis that DA-dependent mechanisms of the DStr are involved in habit learning (Reading et al, 1991; White, 1996; Faure et al, 2005; Daw and Doya, 2006). Thus, these data support the hypothesis that aspects of ‘drug-seeking’ behavior in response to cues previously associated with abused drugs may have acquired habitual qualities divorced of immediate associations with affective experiences produced by the drug itself.

The finding that NAcc core inactivation decreased all measures of conditioned reinforcement (Table 1) supports previous findings that the integrity of the NAcc core is necessary for CS (cue)-induced reinstatement (Fuchs et al, 2004a), the expression of the effects of conditioned reinforcement, as measured under second-order schedules of reinforcement (Ito et al, 2000; Fattore et al, 2002; Di Ciano and Everitt, 2004b) and the acquisition of novel response with a food-paired conditioned reinforcer (Taylor and Robbins, 1984; Parkinson et al, 1999). The present findings cannot easily be explained in terms of motor deficits, as nose pokes measured in parallel were not affected by inactivation of the NAcc core. In general, these effects of NAcc core and DStr inactivation are consistent with the hypothesis (Everitt and Robbins, 2005) that drug-seeking behavior, for which delays to reinforcement are mediated by drug-paired conditioned reinforcers, recruits progressively more components of the cascading corticostriatal loop circuitry described by Haber et al (2000), in a ventral striatal to dorsal striatal manner.

The precise role of the NAcc shell subregion as the initial node in this circuitry remains unclear. By comparison with effects in the DStr or NAcc core, inactivation of the NAcc shell had no effect on responding for the drug-paired conditioned reinforcer. This is consistent with the lack of effect of shell lesions on responding with conditioned reinforcement per se (Parkinson et al, 1999; Ito et al, 2004), although such lesions do block the stimulant-induced enhancements of responding that occur with food- or drug-paired conditioned reinforcers (Parkinson et al, 1999; Ito et al, 2004). The NAcc shell inactivation actually increased responding in the CS (cue)-induced reinstatement paradigm, supporting the dissociation once more from the effects of NAcc core lesions in the acquisition of a new response for conditioned reinforcement procedure (Parkinson et al, 1999; Ito et al, 2004). The increased responding during CS (cue)-induced reinstatement following shell inactivation is suggestive of an inhibitory influence on the activation of CS–drug or response–drug associations, possibly arising from the environmental context. A possible role for the NAcc shell region in mediating such contextual effects derives from its close anatomical (Van den Bercken and Cools, 1982; Groenewegen et al, 1987; Selden et al, 1990; McDonald and White, 1993; Floresco et al, 2001) associations with the hippocampus, as distinct from the NAcc core region, which receives afferents predominantly from the amygdala (Groenewegen et al, 1999). Moreover, hippocampal and amygdaloid projections to the NAcc often exert competing influences on behavior in response to contextual or discrete cues, respectively (Van den Bercken and Cools, 1982; Floresco et al, 2001), consistent with the opposed effects of NAcc core and NAcc shell inactivations in this experiment.

The present finding that inactivations of the NAcc shell increased CS (cue)-induced reinstatement is not consistent with previous findings that inactivation of the NAcc shell had no effect on CS (cue)-induced reinstatement (Fuchs et al, 2004a). This discrepancy may be related to the fact that in CS (cue)-induced reinstatement the response is maintained by expectancy of both the drug and the CS, while these factors are separated in the present procedure. Thus, in CS (cue)-induced reinstatement, the competing influences of the NAcc core and NAcc shell systems on these separate processes may result in a lack of influence of the NAcc shell in CS (cue)-induced reinstatement. These conjectures are speculative but it is possible that under certain conditions the NAcc shell and NAcc core also exert competing influences on response output, there being some behavioral (Weiner et al, 1996) as well as neuroanatomical evidence (Haber et al, 2000) to support this hypothesis.

Indeed, inactivations of the NAcc shell also decreased stress-induced reinstatement (McFarland et al, 2004) while DA receptor antagonists infused into the NAcc shell decreased cocaine-induced reinstatement (Anderson et al, 2003; Bachtell et al, 2005), suggesting that the NAcc shell may mediate both stress- and cocaine-induced reinstatement through mediation of the interoceptive effects of stress and drug, and specifically through dopaminergic mechanisms. Moreover, DA efflux in the NAcc shell was increased following the presentation to rats of Pavlovian stimuli predictive of food (Bassareo and Di Chiara, 1999), though not self-administered cocaine (Ito et al, 2000). Pavlovian stimuli (Holland and Rescorla, 1975), perhaps in contrast to conditioned reinforcers (Parkinson et al, 2005), encode with them a value of the outcome and thus the NAcc shell, via DA, may be important in behavior that is maintained by interoceptive cues/states.

CONCLUSIONS

In sum, the present study provides evidence for dissociable roles of the DStr, NAcc core, and NAcc shell in persistent responding for a drug-paired conditioned reinforcer and CS (cue)-induced reinstatement. The disruption, following inactivation of the DStr, of the tests used here of conditioned reinforcement suggests that the DStr is involved selectively in habitual behavior controlled by conditioned reinforcers. By comparison, inactivation of the NAcc core decreased all measures of conditioned reinforcement, but did so differently from the DStr. Thus, the NAcc core may control the vigor of responding by serving as a limbic-motor interface (Mogenson et al, 1980) that may, in part, provide discriminative control over behavior directed toward distal goals. The NAcc shell, by comparison, seems important in part for control of behavior by interoceptive cues.