Advertisement

Activity in the dorsomedial striatum underlies serial reversal learning performance under probabilistic uncertainty

Open AccessPublished:August 25, 2022DOI:https://doi.org/10.1016/j.bpsgos.2022.08.005

      ABSTRACT

      Background

      Cortico-striatal circuits, particularly the dorsomedial striatum (DMS) and lateral orbitofrontal cortex, are critical for navigating reversal learning under probabilistic uncertainty. These same areas are implicated in the reversal learning impairments observed in those with psychosis, as well as their psychotic symptoms, suggesting they may share a common neurobiological substrate. To address this question, we used psychostimulant exposure and specific activation of the DMS during reversal learning in mice to assess corticostriatal activity.

      Methods

      We used amphetamine treatment to induce psychosis-relevant neurobiology in male mice during reversal learning and to examine pathway specific cortico-striatal activation. To determine the causal role of DMS activity we used chemogenetics to drive midbrain inputs during a range of probabilistic contingencies.

      Results

      Mice treated with amphetamine showed altered punishment learning, which was associated with decreased shifting after losses and increased perseverative errors after reversals. Reversal learning performance and strategies were dependent on increased activity in lateral orbitofrontal cortex to DMS circuits, as well as in the DMS itself. Specific activation of midbrain to DMS circuits also decreased shifting after losses and reversal learning performance. However, these alterations were dependent on the probabilistic contingency.

      Conclusions

      Our work suggests that the DMS plays a multifaceted role in reversal learning. Increasing DMS activity impairs multiple reversal learning processes dependent on the level of uncertainty, confirming its role in the maintenance and selection of incoming cortical inputs. Together, these outcomes suggest that elevated dopamine in the DMS could contribute to decision-making impairments in those with psychosis.

      Keywords

      INTRODUCTION

      Cognitive flexibility is an essential executive function that allows an organism to appropriately adapt behaviors in response to changes in the external environment. Impairments in cognitive flexibility have been observed in a range of neuropsychiatric disorders including schizophrenia, Parkinson’s disease, obsessive compulsive disorder and substance abuse (

      Everitt BJ, Parkinson JA, Olmstead MC, Arroyo M, Robledo P, Robbins TW (1999): Associative processes in addiction and reward - The role of amygdala-ventral striatal subsystems. In: McGinty JF, editor. Advancing from the Ventral Striatum to the Extended Amygdala: Implications for Neuropsychiatry and Drug Abuse: In Honor of Lennart Heimer. New York: New York Acad Sciences, pp 412-438.

      ,
      • Kehagia A.A.
      • Murray G.K.
      • Robbins T.W.
      Learning and cognitive flexibility: frontostriatal function and monoaminergic modulation.
      ,
      • Murray G.K.
      • Cheng F.
      • Clark L.
      • Barnett J.H.
      • Blackwell A.D.
      • Fletcher P.C.
      • et al.
      Reinforcement and Reversal Learning in First-Episode Psychosis.
      ,
      • Waltz J.A.
      The neural underpinnings of cognitive flexibility and their disruption in psychotic illness.
      ). The pathophysiology of these deficits is commonly investigated via reversal learning paradigms that have been developed for use across species, including humans, non-human primates and rodents (
      • Izquierdo A.
      • Brigman J.L.
      • Radke A.K.
      • Rudebeck P.H.
      • Holmes A.
      The neural basis of reversal learning: An updated perspective.
      ).
      Cognitive dysfunction is a core deficit in individuals with schizophrenia (
      • Keefe R.S.E.
      • Bilder R.M.
      • Harvey P.D.
      • Davis S.M.
      • Palmer B.W.
      • Gold J.M.
      • et al.
      Baseline Neurocognitive Deficits in the CATIE Schizophrenia Trial.
      ). These impairments often include disturbances in decision-making, including difficulties learning from positive and negative feedback that can manifest as maladaptive reward learning (
      • Kesby J.P.
      • Murray G.K.
      • Knolle F.
      Neural circuitry of salience and reward processing in psychosis.
      ). This has been confirmed in multiple studies which demonstrate that people with schizophrenia achieve significantly fewer reversals than healthy controls in reversal learning tasks (
      • Murray G.K.
      • Cheng F.
      • Clark L.
      • Barnett J.H.
      • Blackwell A.D.
      • Fletcher P.C.
      • et al.
      Reinforcement and Reversal Learning in First-Episode Psychosis.
      ,
      • Culbreth A.J.
      • Gold J.M.
      • Cools R.
      • Barch D.M.
      Impaired Activation in Cognitive Control Regions Predicts Reversal Learning in Schizophrenia.
      ,
      • Waltz J.A.
      • Gold J.M.
      Probabilistic reversal learning impairments in schizophrenia: Further evidence of orbitofrontal dysfunction.
      ,
      • Suetani S.
      • Baker A.
      • Garner K.
      • Cosgrove P.
      • Mackay-Sim M.
      • Siskind D.
      • et al.
      Impairments in goal-directed action and reversal learning in a proportion of individuals with psychosis.
      ). Behavioral impairments are often paired with a reduction in the use of optimal trial-by-trial strategies, such as win-stay (selecting the same stimulus after a ‘win’) and lose-shift (selecting the opposing stimulus after a ‘loss’), implemented by individuals to aid in task navigation (
      • Culbreth A.J.
      • Gold J.M.
      • Cools R.
      • Barch D.M.
      Impaired Activation in Cognitive Control Regions Predicts Reversal Learning in Schizophrenia.
      ). Impairments in reversal learning have been observed in both the initial discrimination phase and the subsequent reversal phases (
      • Culbreth A.J.
      • Gold J.M.
      • Cools R.
      • Barch D.M.
      Impaired Activation in Cognitive Control Regions Predicts Reversal Learning in Schizophrenia.
      ,
      • Schlagenhauf F.
      • Huys Q.J.
      • Deserno L.
      • Rapp M.A.
      • Beck A.
      • Heinze H.J.
      • et al.
      Striatal dysfunction during reversal learning in unmedicated schizophrenia patients.
      ,
      • Weiler J.A.
      • Bellebaum C.
      • Brüne M.
      • Juckel G.
      • Daum I.
      Impairment of probabilistic reward-based learning in schizophrenia.
      ,
      • Murray G.K.
      • Cheng F.
      • Clark L.
      • Barnett J.H.
      • Blackwell A.D.
      • Fletcher P.C.
      • et al.
      Reinforcement and reversal learning in first-episode psychosis.
      ,
      • Waltz J.A.
      • Kasanova Z.
      • Ross T.J.
      • Salmeron B.J.
      • McMahon R.P.
      • Gold J.M.
      • et al.
      The roles of reward, default, and executive control networks in set-shifting impairments in schizophrenia.
      ).
      The etiology of schizophrenia and its accompanying cognitive deficits is still unknown, but evidence highlights a role for subcortical dopamine systems in reward learning and reinforcement (
      • Kesby J.P.
      • Murray G.K.
      • Knolle F.
      Neural circuitry of salience and reward processing in psychosis.
      ). An area of particular interest is the associative striatum (dorsomedial striatum [DMS] in rodents (
      • Conn K.
      • Burne T.H.J.
      • Kesby J.P.
      Subcortical dopamine and cognition in schizophrenia: looking beyond psychosis in preclinical models.
      )), which is the site of increased dopamine function in psychosis and an important area in goal-directed action and decision-making (
      • Kesby J.P.
      • Eyles D.W.
      • McGrath J.J.
      • Scott J.G.
      Dopamine, psychosis and schizophrenia: the widening gap between basic and clinical neuroscience.
      ). Functional neuroimaging studies in healthy subjects support a role for subcortical dopamine in reversal learning. For example, increased dopamine receptor availability in the caudate nucleus (associative striatum) has been shown to correlate with more reversal learning errors (
      • Clatworthy P.L.
      • Lewis S.J.
      • Brichard L.
      • Hong Y.T.
      • Izquierdo D.
      • Clark L.
      • et al.
      Dopamine release in dissociable striatal subregions predicts the different effects of oral methylphenidate on reversal learning and spatial working memory.
      ).
      Typically, probabilistic reversal learning (PRL) paradigms are used to study cognitive flexibility, with stimuli presented as both high (80%) and low (20%) probability reward outcomes (80:20 contingency). This requires subjects to make decisions in the presence of misleading feedback, allowing accumulated evidence across previous trials to guide choices. Once a set performance criterion has been achieved, the high and low rewarded stimuli are reversed, prompting the subject to adjust their behavior based on reward-outcome feedback. Numerous studies, both clinical and preclinical, have identified the importance of, and the complex relationship between, the striatum and the orbitofrontal cortex during the serial reversal learning phases of PRL tasks (
      • Kesby J.P.
      • Murray G.K.
      • Knolle F.
      Neural circuitry of salience and reward processing in psychosis.
      ,
      • Dalton G.L.
      • Wang N.Y.
      • Phillips A.G.
      • Floresco S.B.
      Multifaceted Contributions by Different Regions of the Orbitofrontal and Medial Prefrontal Cortex to Probabilistic Reversal Learning.
      ,
      • Robinson O.J.
      • Frank M.J.
      • Sahakian B.J.
      • Cools R.
      Dissociable responses to punishment in distinct striatal regions during reversal learning.
      ,
      • Ragozzino M.E.
      The contribution of the medial prefrontal cortex, orbitofrontal cortex, and dorsomedial striatum to behavioral flexibility.
      ,
      • Clarke H.F.
      • Robbins T.W.
      • Roberts A.C.
      Lesions of the medial striatum in monkeys produce perseverative impairments during reversal learning similar to those produced by lesions of the orbitofrontal cortex.
      ,
      • Clarke H.F.
      • Cardinal R.N.
      • Rygula R.
      • Hong Y.T.
      • Fryer T.D.
      • Sawiak S.J.
      • et al.
      Orbitofrontal dopamine depletion upregulates caudate dopamine and alters behavior via changes in reinforcement sensitivity.
      ,
      • Izquierdo A.
      Functional Heterogeneity within Rat Orbitofrontal Cortex in Reward Learning and Decision Making.
      ). However, despite the clinical relevance of reversal learning, low performance levels in rodents (
      • Milienne-Petiot M.
      • Kesby J.P.
      • Graves M.
      • van Enkhuizen J.
      • Semenova S.
      • Minassian A.
      • et al.
      The effects of reduced dopamine transporter function and chronic lithium on motivation, probabilistic learning, and neurochemistry in mice: Modeling bipolar mania.
      ,
      • Metha J.A.
      • Brian M.L.
      • Oberrauch S.
      • Barnes S.A.
      • Featherby T.J.
      • Bossaerts P.
      • et al.
      Separating Probability and Reversal Learning in a Novel Probabilistic Reversal Learning Task for Mice.
      ) and the complex role of the DMS have constrained prior studies.
      In the present study, we aimed to 1) establish a PRL task in mice that features more reversals/session, and above-chance performance at a range of probabilistic reward contingencies; 2) identify how amphetamine treatment alters reversal learning and cortico-striatal network activation; and 3) chemogenetically activate midbrain inputs to the DMS to determine its specific role in reversal learning under a variety of probabilistic contingencies. Our work highlights a multifaceted role for the DMS in reversal learning.

      METHODS AND MATERIALS

      Animals

      For detailed protocols see the Supplementary methods. 10-week old male C57BL/6JArc mice (Animal Resources Centre, WA, Australia), were used (31 for Experiment 1, and 23 for Experiment 2). All procedures were performed with the approval from the University of Queensland Animal Ethics Committee (QBI/079/17).

      PRL protocol

      PRL testing was conducted in eight Plexiglas operant chambers (model ENV-307A, Med Associates Inc.). The chambers featured two nose-poke holes and a reward magazine located on the opposite wall, which could deliver strawberry milk. The target nose-poke was defined as that with the highest reward rate. No punishment followed an unrewarded response, rather punishment/loss reflects a lack of expected reward. After 6 consecutive target nose pokes, the target and non-target reward contingencies were reversed. Behavioural measures included the total number of trials, latency to respond, number of reversals/100 trials, proportion of win-stay and lose-shift strategies, and number of perseverative errors (consecutive errors after a reversal).

      Experiment 1: Amphetamine treatment and corticostriatal networks

      See Figure 1 (top panel) for a detailed experimental timeline.
      Figure thumbnail gr1
      Figure 1Timeline of experimental manipulations. Top panel: Experiment 1. Mice were trained until PRL performance was stable and then underwent tract-tracing surgery. After recovery and re-baselining, mice were tested for each contingency (80:20, 70:30, 80:40 and 90:10), separated by 2 days. Following 5 further days of baseline PRL testing, mice were administered amphetamine (1 mg/kg, i.p.) or saline (15 min prior to testing) at an 80:20 contingency. Mice were perfused 60 min after completion of testing (i.e., 90 min after the middle of the reversal learning session) for subsequent cFos analysis. Bottom panel: Experiment 2. After DREADDs surgery and PRL training, mice underwent acute clozapine-N-oxide (CNO; AK Scientific) administration (30 min prior to testing). This was conducted using a within-subject Latin-square design at the 80:20 contingency with CNO (0, 0.5, 1 and 2 mg/kg, in 0.5% dimethyl sulfoxide [DMSO] and 0.9% saline). This was followed by a crossover design with CNO (0 and 2 mg/kg) at the 80:40 contingency. Subsequently, mice were given daily fresh water (0.25% DMSO) for 5 days at 80:20 followed by a water solution containing CNO (∼8mg/kg/day). CNO exposure was continued for 23 days; 14 days at 80:20, 4 days at 80:40 and 5 days at 70:30. After testing was complete, a small cohort of mice were administered 2 mg/kg CNO i.p. and perfused 2 h later for cFos analysis.

      Surgery and tract tracing procedures

      Cholera toxin subunit B (CTb) conjugated with Alexa Fluor 555 (C-34776, Thermo Fisher Scientific) was injected into the right posterior DMS (0.5μl, AP+0.01; ML-0.18; DV-0.29 [from the skull surface], in mm relative to bregma (

      Paxinos G, Franklin K (2001): The Mouse Brain in Stereotaxic Coordinates. 2nd ed. San Diego: Academic Press.

      )) and a CTb conjugated with Alexa Fluor 647 (C-34778, Thermo Fisher Scientific) was injected into the left anterior DMS (0.5μl, AP+0.11; ML+0.145; DV-0.29).

      Simulation and computational modelling

      Simulations (5000 simulations of 500 trials) with random choices were coded in R (version 3.6) to quantify the likelihood of ‘chance’ reversals. We modelled latent task variables using the hBayesDM package for R (
      • Ahn W.Y.
      • Haines N.
      • Zhang L.
      Revealing Neurocomputational Mechanisms of Reinforcement Learning and Decision-Making With the hBayesDM Package.
      ). A reward/punishment (RP) learning model with parameters for reward learning rate, punishment learning rate, and inverse temperature was selected (
      • den Ouden H.E.
      • Daw N.D.
      • Fernandez G.
      • Elshout J.A.
      • Rijpkema M.
      • Hoogman M.
      • et al.
      Dissociable effects of dopamine and serotonin on reversal learning.
      ).

      Histology, immunofluorescence and cFos quantification

      Brain sections were stained for cFos and DAPI (see Table S1 for details). Sections were imaged on a spinning-disk confocal system and 40x objective. Image acquisition was performed using SlideBook 6.0 (3I, Inc). Image preparation and regional masks (

      Paxinos G, Franklin K (2001): The Mouse Brain in Stereotaxic Coordinates. 2nd ed. San Diego: Academic Press.

      ) were completed using ImageJ (
      • Schneider C.A.
      • Rasband W.S.
      • Eliceiri K.W.
      NIH Image to ImageJ: 25 years of image analysis.
      ). Nuclei were segmented with Cellpose (
      • Stringer C.
      • Wang T.
      • Michaelos M.
      • Pachitariu M.
      Cellpose: a generalist algorithm for cellular segmentation.
      ) and fed into Cellprofiler v4.1 for quantification (
      • Stirling D.R.
      • Swain-Bowden M.J.
      • Lucas A.M.
      • Carpenter A.E.
      • Cimini B.A.
      • Goodman A.
      CellProfiler 4: improvements in speed, utility and usability.
      ). cFos+ cells were determined as those with >1.1-fold mean cFos intensity when compared to the mean intensity of all cells per image.

      Experiment 2: Acute and chronic activation of midbrain to DMS cells with DREADDs

      See Figure 1 (bottom panel) for a detailed experimental timeline.

      Surgery for pathway-specific DREADDs

      Surgery was performed as for Experiment 1. Transfection of striatal pathways was achieved by bilaterally microinjecting Cre-dependent hM3D(Gq) flex-switch DREADDs virus (pAAV5-hSyn-DIO-hM3D(Gq)-mCherry; Addgene; 0.5 μl) into the substantia nigra (AP-3.10; ML±1.25; DV-4.20) and a retrograde Cre-recombinase virus (pENN.AAV.hSyn.HI.eGFP-Cre.WPRE.SV40; Addgene; 0.7μl) into the DMS (AP+0.11; ML±1.25; DV-2.90). A control virus not expressing DREADDs (pAAV5-hSyn-DIO- mCherry; Addgene) was used for half of the animals. From 23 mice, strong viral expression in both hemispheres was confirmed in 12 control and 6 DREADDs mice.

      Histology and cFos quantification

      Sectioning and histology were performed as for Experiment 1 (see Table S1 for antibody details). Striatal sections were stained for mCherry and tyrosine hydroxylase. Midbrain sections were stained for mCherry and cFos. Images were taken at 20x magnification.

      General statistics

      Analyses were performed with IBM SPSS Statistics 26 and R (v4.1.2; 32). Data were analyzed using analysis of variance (ANOVA) and repeated-measures ANOVAs when within-subject factors were present. Post hoc comparisons were performed using Šídák corrections. Graphical results are expressed as mean ± standard error of the mean (SEM).

      RESULTS

      A minimum level of probabilistic uncertainty is required to impair performance

      By optimizing task parameters, we were able to demonstrate that mice can perform PRL across a range of contingencies. Mice performed the task well above chance, even at more difficult contingencies (Figure 2A). The total number of trials completed was not affected by differing contingencies (Figure 2B), but a main effect of contingency on the number of reversals/100 trials was observed (F3,88=19.5, p<0.001; Figure 2C). Mice achieved fewer reversals at all contingencies compared to the 90:10 contingency (p<0.001). This suggests that 90:10 contingencies do not provide much probabilistic uncertainty, and the 80:20 contingency is an effective probabilistic contingency for use in mice (and the most common in humans (
      • Izquierdo A.
      • Brigman J.L.
      • Radke A.K.
      • Rudebeck P.H.
      • Holmes A.
      The neural basis of reversal learning: An updated perspective.
      )). This is further supported by the lack of differences in reversal performance when comparing deterministic (100:0, 6.45±0.26 reversals/100 trials) with 90:10 contingencies (6.15±0.30 reversals/100 trials). Alterations in contingencies also affected choice strategy, with a significant difference in win-stay probability being observed (F3,88=3.5, p<0.05; Figure 2D). Mice had a decrease in win-stay probability during the 80:40 contingency compared with the 90:10 contingency (p<0.05), commensurate with the increased misleading feedback for 80:40. In contrast, lose-shift probability was not significantly different between contingencies (Figure 2E). This confirms that mice are capable of navigating PRL well above chance when task parameters are optimized. We also assessed stability over 5 days at 80:20 (Figure S1), with mice demonstrating consistent performance and low variance in win-stay and lose-shift probability. These data parallel studies in humans highlighting that probabilistic reversal learning tasks have high retest stability (
      • Waltmann M.
      • Schlagenhauf F.
      • Deserno L.
      Sufficient reliability of the behavioral and computational readouts of a probabilistic reversal learning task.
      ). Taken together, this task protocol allows for within-session manipulations and the use of more uncertain/difficult contingencies in preclinical rodent studies.
      Figure thumbnail gr2
      Figure 2Mouse performance when compared with chance at various contingencies. The number of reversals (as a proportion of all mice/simulations) achieved during the contingency modifications (A). The probability of being rewarded on the high:low lever was tested at 90:10, 80:20, 80:40 and 70:30 (N=23). Mice performed well above chance when given the 90:10 and 80:20 contingencies (Note: x-axis is not linear). Even at the more difficult 80:40 and 70:30 contingencies, >90% of mice performed above chance levels. Mice maintained a high number of trials per session under all contingencies (B). As the difference between high vs. low decreased (80:40 and 70:30) mice were less efficient at completing reversals (C). Win-stay probability (D) was significantly greater at 90:10 compared with 80:40, whereas lose-shift probability (E) was not significantly affected. Data are expressed as mean ± standard error. *p<0.05, ***p<0.001

      Amphetamine treatment alters punishment learning

      Psychomimetic drugs such as amphetamine are often used to model the increased dopaminergic function associated with psychosis (
      • Kesby J.P.
      • Eyles D.W.
      • McGrath J.J.
      • Scott J.G.
      Dopamine, psychosis and schizophrenia: the widening gap between basic and clinical neuroscience.
      ,

      Angrist B (1994): Amphetamine psychosis: clinical variations of the syndrome. In: Cho AK, Segal DS, editors. Amphetamine and its analogues. San Diego: Academic Press, pp 387-414.

      ,
      • Krystal J.H.
      • Perry Jr., E.B.
      • Gueorguieva R.
      • Belger A.
      • Madonick S.H.
      • Abi-Dargham A.
      • et al.
      Comparative and interactive human psychopharmacologic effects of ketamine and amphetamine: implications for glutamatergic and dopaminergic model psychoses and cognitive function.
      ,
      • Robinson T.E.
      • Becker J.B.
      Enduring changes in brain and behavior produced by chronic amphetamine administration: a review and evaluation of animal models of amphetamine psychosis.
      ,
      • Abi-Dargham A.
      • van de Giessen E.
      • Slifstein M.
      • Kegeles L.S.
      • Laruelle M.
      Baseline and Amphetamine-Stimulated Dopamine Activity Are Related in Drug-Naive Schizophrenic Subjects.
      ,
      • Lipska B.K.
      • Jaskiw G.E.
      • Weinberger D.R.
      Postpubertal Emergence of Hyperresponsiveness to Stress and to Amphetamine after Neonatal Excitotoxic Hippocampal Damage - a Potential Animal-Model of Schizophrenia.
      ,
      • Petty A.
      • Cui X.
      • Ali A.
      • Du Z.
      • Srivastav S.
      • Kesby J.P.
      • et al.
      Positive symptom phenotypes appear progressively in "EDiPS", a new animal model of the schizophrenia prodrome.
      ,
      • Laruelle M.
      • Abi-Dargham A.
      • Gil R.
      • Kegeles L.
      • Innis R.
      Increased dopamine transmission in schizophrenia: Relationship to illness phases.
      ). However, increasing brain dopaminergic function did not alter the number of reversals/100 trials (Figure 3A), the response latency (Figure 3B) or win-stay probability (Figure 3C) during reversal learning. But amphetamine treatment led to a decrease in lose-shift probability (F1,19=10.4, p<0.01; Figure 3D) and increase the number of perseverative errors after a reversal (F1,19=5.2, p<0.05; Figure 3E). Computational modelling revealed that amphetamine significantly decreased the punishment learning parameter (F1,19=5.5, p<0.05; Figure 3G), suggesting that punishment learning was biased toward past outcomes. This parameter change infers that the mouse is placing less weight on recent losses (lack of a reward in this protocol) and is relying more on the accrual of losses over time. This fits with the observed decreases in lose-shift probability after the most recent loss, and slow internal updating after a reversal (i.e., increased perseverative errors). No differences were observed for reward learning (Figure 3F) or inverse temperature (Figure 3H) parameters. These outcomes demonstrate that amphetamine treatment selectively alters learning associated with loss (or the absence of a reward) in mice. Therefore, amphetamine-induced impairments in reversals/100 trials may be more likely on contingencies with greater levels of misleading losses (i.e., 70:30 etc.).
      Figure thumbnail gr3
      Figure 3Effects of acute amphetamine challenge on task performance. Mice were administered 1 mg/kg amphetamine (N=10) or saline (N=11) prior to testing at the 80:20 contingency. Amphetamine did not significantly alter the efficiency of completing reversals (A), the average trials to reach criterion for serial reversal learning (B), or win-stay probability (C). Amphetamine-treated mice were significantly less likely to shift after a loss (D) and displayed higher rates of perseveration (Persev) after a reversal (E). Computational modelling using the RP model (F-H) indicated that amphetamine increased the bias toward past negative outcomes (lack of reward) over recent ones (G) without altering reward learning (F) or deterministic behavior (inverse temperature; H). Data are expressed as mean ± standard error. *p<0.05, **p<0.01
      Amphetamine treatment increases activity in the DMS, and lateral orbitofrontal cortex inputs to the DMS, during reversal learning
      We were interested in the role of the DMS and its corticostriatal inputs during reversal learning and after amphetamine. To identify how activity in these corticostriatal circuits modulate PRL we next quantified cFos expression in combination with CTb retrograde labelling of DMS inputs (Figure 4A). We determined the percentage of cFos+ cells and their intensity distribution in a range of corticostriatal regions (Figure 4B-D). As others have observed for these corticostriatal circuits (
      • Heilbronner S.R.
      • Rodriguez-Romaguera J.
      • Quirk G.J.
      • Groenewegen H.J.
      • Haber S.N.
      Circuit-Based Corticostriatal Homologies Between Rat and Primate.
      ,
      • Mailly P.
      • Aliane V.
      • Groenewegen H.J.
      • Haber S.N.
      • Deniau J.M.
      The rat prefrontostriatal system analyzed in 3D: evidence for multiple interacting functional units.
      ,
      • Hunnicutt B.J.
      • Jongbloets B.C.
      • Birdsong W.T.
      • Gertz K.J.
      • Zhong H.
      • Mao T.
      A comprehensive excitatory input map of the striatum reveals novel functional organization.
      ), innervation of the DMS was greatest from the anterior cingulate cortex (ACC), ventral orbitofrontal cortex (VO) and lateral orbitofrontal cortex (LO), with >75% of CTb labelled cells originating in these areas (Figure 4E). Of the regions assessed, only the DMS had a significantly greater number of cFos+ cells after amphetamine treatment (Figure 4H; t19=2.5, p<0.05, and see Table S2). Other studies looking at cFos+ cell number after amphetamine have also observed specific increases in the DMS (
      • Ren K.
      • Guo B.
      • Dai C.
      • Yao H.
      • Sun T.
      • Liu X.
      • et al.
      Striatal Distribution and Cytoarchitecture of Dopamine Receptor Subtype 1 and 2: Evidence from Double-Labeling Transgenic Mice.
      ), suggesting that the DMS is more susceptible to activity-induced changes after amphetamine treatment.
      Figure thumbnail gr4
      Figure 4Corticostriatal cFos activation during reversal learning and amphetamine treatment. Fluorescently labelled CTb was injected into the DMS to retrogradely label cortical inputs to the DMS (A). cFos expression in CTb-labeled and -unlabeled cells was quantified in cortical and striatal areas (B-D). The ACC and VO/LO areas had the largest proportion of CTb-labeled cells (E), highlighting their dense connectivity to the DMS. Percentage of cFos+ cells in corticostriatal inputs (ACC and LO) to the DMS, and the DLS (F-I). Amphetamine treatment only increased the percentage of cFos+ cells in the DMS. There was significant right-shift in cFos+ cell intensity levels (fold-increase from average of all cells) after amphetamine in ACC cells projecting to the DMS, and in the DMS itself (J-M). Amphetamine increased the average cFos intensity in LO cells projecting to the DMS (inset) and in the DMS (inset), suggesting greater cFos activation. Cumulative distribution plots (N-Q) of cells in the ACC and LO projecting to the DMS show significantly greater increases in cFos intensity after amphetamine treatment. Similarly, amphetamine increased cFos intensity in the DMS, but not the DLS. ACC, Anterior cingulate cortex; PL, prelimbic cortex; IL, infralimbic cortex; MO, medial orbitofrontal cortex; VO, ventral orbitofrontal cortex; LO, lateral orbitofrontal cortex; DLS, dorsolateral striatum; AcbC, nucleus accumbens core; AcbSh, nucleus accumbens shell; eCDF, empirical cumulative distribution function; KS, Kolmogorov-Smirnov test. Data are expressed as mean ± standard error (N=8-11). *p<0.05, **p<0.01
      In contrast to cFos+ cell percentage, amphetamine treatment increased the intensity of cFos expression in multiple areas (Table S2 and S3). For example, all cortical areas showed a significant shift to the right of the intensity histograms (significant Bin x Drug interaction), although the large number of cells quantified suggests that this reflects a small real-world increase in intensity. For the corticostriatal inputs (CTb-labelled cells), the ACC showed a significant Bin x Drug interaction (Figure 4J; F17,289=3.6, p<0.01) and the LO had a significantly higher mean cFos intensity (Figure 4K inset; t13=3.4, p<0.01). In the striatum, a significant interaction of Bin x Drug (right-shifted curve) was observed in the DMS (Figure 4L; F17,323=3.9, p<0.05) but not in the dorsolateral striatum (DLS) (Figure 4M) or nucleus accumbens (Acb) subregions (Table S3). Significant Kolmogorov-Smirnov tests of the cumulative distribution frequencies supported increased cFos intensities in the ACC and LO inputs to the DMS (Figure 4N-O). Together these results demonstrate that amphetamine alters activity in the DMS and corticostriatal projections from the ACC and LO to the DMS. Furthermore, that the level of cFos expression used to classify cFos+ cells may have dramatic effects on study outcomes.

      Reversal learning performance is associated with LO and DMS corticostriatal networks

      Having identified specific differences in cFos expression within the ACC and LO inputs to the DMS, we were interested in determining how these relate to behavioral performance. We considered saline- and amphetamine-treated mice as a single group, given we do not know their individual dopamine levels and amphetamine-induced cortical dopamine release is modest (∼2 fold of baseline levels)(
      • Ventura R.
      • Alcaro A.
      • Cabib S.
      • Conversi D.
      • Mandolesi L.
      • Puglisi-Allegra S.
      Dopamine in the medial prefrontal cortex controls genotype-dependent effects of amphetamine on mesoaccumbens dopamine release and locomotion.
      ,
      • Grant A.
      • Hoops D.
      • Labelle-Dumais C.
      • Prévost M.
      • Rajabi H.
      • Kolb B.
      • et al.
      Netrin-1 receptor-deficient mice show enhanced mesocortical dopamine transmission and blunted behavioural responses to amphetamine.
      ). Especially considering that studies in humans have indicated that manipulating striatal dopamine levels can differentially alter reward and punishment learning depending on baseline dopamine turnover (
      • Cools R.
      • Frank M.J.
      • Gibbs S.E.
      • Miyakawa A.
      • Jagust W.
      • D'Esposito M.
      Striatal dopamine predicts outcome-specific reversal learning and its sensitivity to dopaminergic drug administration.
      ). Moreover, we were interested in how corticostriatal activity relates to behavioral performance more broadly. We therefore used a principal components analysis (PCA) to identify which cFos measures (ACC and LO inputs, as well as striatal outcomes) were associated with key performance outcomes (i.e., sharing strong loadings). This resulted in a four-factor solution accounting for 78% of the overall variance (Figure 5A). Factor 1 indicated strong relationships between LO inputs to the DMS, the DMS itself and behavioral outcomes. cFos intensity in these areas was positively associated with reversal performance and win-stay probability, but negatively associated with lose-shift probability. Factor 2 grouped ACC and LO function, suggesting a positive relationship between cortical activity in the ACC and LO independent of behavioral outcomes. Factor 3 indicated that activity in the nucleus accumbens is positively associated with win-stay probability, whereas Factor 4 indicated that a negative relationship exists between DLS activity and lose-shift probability. The outcomes in Factor 1 were supported by significant correlations between the mean cFos intensity of DMS cells with inputs from the LO (Figure 5B), reversal performance (Figure 5C), win-stay probability (Figure 5D) and lose-shift probability (Figure 5E). Taken together, this demonstrates that LO inputs to the DMS and activity within the DMS itself are associated with multiple aspects of PRL. Systemic amphetamine compromises learning after losses but also drives greater cortico-striatal recruitment and activity which may help negate broad impairments in reversal performance.
      Figure thumbnail gr5
      Figure 5Principal component analysis of corticostriatal cFos activity and behavior. Loading heatmaps for principal component analysis of ACC, LO and striatal cFos measures and key behavioral outcomes (A). Four factors were identified that accounted for nearly 80% of the variance (Eigenvalues; Factor 1 = 30.8%, Factor 2 = 23.2%, Factor 3 = 13.0% and Factor 4 = 10.0%). Factor 1 captured LO inputs to the DMS and all three behavioral measures of interest. Factor 2 captured primarily cortical activity. Factor 3 included activity in the nucleus accumbens and win-stay probability. Factor 4 included DLS activity and lose-shift probability. Relevant correlations with DMS cFos intensity, activity of LO inputs to the DMS and behavior (B-E). There was a strong correlation between the cFos intensity of LO cells projecting to the DMS (CTb labelled) and the cFos intensity of DMS cells (B), suggesting LO inputs are synchronized with DMS activity. As highlighted in the PCA, increasing DMS cFos intensity positively correlated with reversal performance (C) and win-stay probability (D), but negatively correlated with lose-shift probability (E). *p<0.05, **p<0.01

      Activation of midbrain to DMS cells with DREADDs

      There is a strong relationship between orbitofrontal cortex and DMS function in reinforcement learning, particularly for dopaminergic systems (
      • Clarke H.F.
      • Robbins T.W.
      • Roberts A.C.
      Lesions of the medial striatum in monkeys produce perseverative impairments during reversal learning similar to those produced by lesions of the orbitofrontal cortex.
      ,
      • Clarke H.F.
      • Cardinal R.N.
      • Rygula R.
      • Hong Y.T.
      • Fryer T.D.
      • Sawiak S.J.
      • et al.
      Orbitofrontal dopamine depletion upregulates caudate dopamine and alters behavior via changes in reinforcement sensitivity.
      ). We were interested in whether increased DMS activity, independent of altered LO inputs, could lead to the observed reversal learning phenotypes. Therefore, we used pathway-specific DREADDs to activate midbrain to DMS projections (Figure 6A), hypothesizing that if increased DMS activity is a critical factor (downstream of LO activity or otherwise) then we should observe a behavioral phenotype akin to that after amphetamine treatment. DREADDs can be activated chronically via CNO in drinking water (
      • Urban D.J.
      • Zhu H.
      • Marcinkiewcz C.A.
      • Michaelides M.
      • Oshibuchi H.
      • Rhea D.
      • et al.
      Elucidation of The Behavioral Program and Neuronal Network Encoded by Dorsal Raphe Serotonergic Neurons.
      ) allowing us to manipulate these circuits acutely and chronically during reversal learning. We confirmed that DREADDs receptors remained effective following all testing. The number of cFos+ cells was significantly greater in the mCherry+ cells of hM3Dq mice compared with controls (Figure 6B; t3=4.0, p<0.05). For cFos intensity there was a significant Cell x Group interaction (Figure 6C; F1,3=26.2, p<0.05), with intensity greater in hM3Dq mCherry+ cells compared with control mCherry+ cells (p<0.05) and hM3Dq unlabeled cells (p<0.01). These results confirm prior studies demonstrating the longevity of DREADDs activity after chronic activation (
      • Urban D.J.
      • Zhu H.
      • Marcinkiewcz C.A.
      • Michaelides M.
      • Oshibuchi H.
      • Rhea D.
      • et al.
      Elucidation of The Behavioral Program and Neuronal Network Encoded by Dorsal Raphe Serotonergic Neurons.
      ).
      Figure thumbnail gr6
      Figure 6Effects of acute stimulation of midbrain projections to the DMS on reversal learning performance. Pathway-specific DREADDs were used to express hM3Dq receptors (or mCherry alone for Controls) in midbrain cells projecting to the DMS (A). Administration of CNO increased cFos+ cell number (B) and cFos intensity (C) in mCherry-labelled cells in the midbrain of hM3Dq-expressing mice (N=2-3). Acute within-subject treatment with CNO (Controls N=12, hM3Dq N=6) did not alter the number of completed trials (D), reversal performance (E) or win-stay probability (F) at 80:20 or 80:40 contingencies. Activating cells projecting from the midbrain to the DMS significantly decreased lose-shift probability (G) at the 80:40 contingency. Data are expressed as mean ± standard error. *p<0.05, **p<0.01

      Acute activation decreases lose-shift probability

      As for amphetamine treatment, acute DREADDs activation did not significantly alter the number of trials completed (Figure 6D), reversal performance (Figure 6E) or win-stay probability (Figure 6F) at 80:20 or 80:40 contingencies. Lose-shift probability was not significantly altered by DREADDs activation at 80:20 (Figure 6G), but at 80:40 there was a significant interaction of Dose x group (F1,16=5.8, p<0.05), with CNO administration in hM3Dq mice significantly decreasing lose-shift probability compared to controls (p<0.05) and vehicle treatment (p<0.01). This result mimicked our observations after amphetamine, but at a more difficult contingency (i.e., 80:40 vs. 80:20). It is also consistent with our recent work on reversal learning in those with early psychosis (

      Suetani S, Baker A, Garner K, Cosgrove P, Mackay-Sim M, Siskind D, et al. (2021): Impairments in goal-directed action and reversal learning in a proportion of individuals with psychosis: evidence for differential phenotypes in early and persistent psychosis. medRxiv.2021.2008.2031.21262937.

      ) showing specific decreases in lose-shift probability at more uncertain contingencies (i.e., 80:40).

      Chronic activation independently impairs reversal performance and lose-shift probability

      The DMS is important for action selection and maintaining optimal strategy use in reversal learning (
      • Conn K.
      • Burne T.H.J.
      • Kesby J.P.
      Subcortical dopamine and cognition in schizophrenia: looking beyond psychosis in preclinical models.
      ,
      • Kesby J.P.
      • Eyles D.W.
      • McGrath J.J.
      • Scott J.G.
      Dopamine, psychosis and schizophrenia: the widening gap between basic and clinical neuroscience.
      ,
      • Ragozzino M.E.
      The contribution of the medial prefrontal cortex, orbitofrontal cortex, and dorsomedial striatum to behavioral flexibility.
      ). Therefore, well trained animals may rely on learnt strategies to maintain performance after acute DMS manipulations. Chronic activation may induce performance alterations by gradually impairing the navigation of reversal learning. We used CNO in drinking water to chronically activate midbrain to DMS pathways during daily PRL testing. This protocol did not affect basic operant outcomes such as the number of trials completed (Figure 7B) or response latency (data not shown). In contrast, for reversal performance (Figure 7C) there was a significant effect of Stage (F3,45=16.2, p<0.001) and a significant interaction of Stage x Group (F3,45=3.8, p<0.05). hM3Dq mice completed fewer reversals/100 trials compared with controls at the 80:40 (p<0.05) and 70:30 (p<0.01) contingencies. There were no significant effects of DREADDS activation on win-stay probability (Figure 7D) but we did observe a significant effect of Stage (F3,45=4.3, p<0.01) and a significant interaction of Stage x Group (F3,45=3.1, p<0.05) for lose-shift probability (Figure 7E). Controls maintained baseline levels of lose-shift probability at all contingencies, whereas hM3Dq mice had a significantly lower lose-shift probability at 80:20 (p<0.05) and 80:40 (p<0.05) compared with baseline. Together these results indicate that lose-shift probability is sensitive to DMS activation, and DMS activation produces a phenotype akin to acute amphetamine treatment. However, DMS activation can also impair reversal performance, which tracks with increasing probabilistic uncertainty.
      Figure thumbnail gr7
      Figure 7Effects of chronic stimulation of midbrain projections to the DMS on reversal learning performance. Experimental timeline featuring 5 days of baseline testing at 80:20 with new water bottles containing DMSO for acclimatization (A). Mice were then given water bottles containing CNO and assessed for a further 14 days at 80:20, followed by 4 days at 80:40 and 5 days at 70:30. All mouse averages are represented as a percentage of their original groups baseline (e.g., Controls normalized to Control baseline [N=11], and hM3Dq normalized to hM3Dq baseline [N=6]). Chronic CNO treatment to stimulate midbrain to DMS pathways did not alter the average number of trials completed at any contingency (B). DREADDs activation during both the 80:40 and 70:30 contingencies significantly decreased reversal performance (C). No differences were observed for win-stay probability (D), but DREADDs activation significantly decreased lose-shift probability during both the 80:20 and 80:40 contingencies (E). Data are expressed as mean ± standard error. *p<0.05, **p<0.01, #p<0.05 compared with hM3Dq baseline.

      DISCUSSION

      In this study we investigated the relationship between corticostriatal activity and reversal learning after dopaminergic manipulations in the DMS of male mice. Our results demonstrate that the DMS regulates multiple processes in reversal learning. Amphetamine treatment altered punishment learning and increased activity in LO inputs to the DMS and the DMS itself. We then confirmed that amphetamine-induced phenotypes can be replicated via specific activation of midbrain to DMS pathways. Moreover, chronic activation of midbrain to DMS pathways resulted in a decline in reversal performance when probabilistic uncertainty is increased. These outcomes suggest that DMS dopamine dysfunction may contribute to reversal learning deficits in disorders such as psychosis and highlights a complex interplay between probabilistic uncertainty and DMS function.

      Serial PRL in mice

      Previous studies have suggested that mice perform PRL tasks at levels at or slightly above chance (
      • Metha J.A.
      • Brian M.L.
      • Oberrauch S.
      • Barnes S.A.
      • Featherby T.J.
      • Bossaerts P.
      • et al.
      Separating Probability and Reversal Learning in a Novel Probabilistic Reversal Learning Task for Mice.
      ), highlighting the need for improved task protocols. In our optimized PRL paradigm, more than 90% of mice performed above chance at complex contingencies (i.e., 80:40, 70:30). Mice maintained levels of win-stay and lose-shift probability comparable to those reported in rats, however win-stay probability was lower than in humans (
      • Dalton G.L.
      • Wang N.Y.
      • Phillips A.G.
      • Floresco S.B.
      Multifaceted Contributions by Different Regions of the Orbitofrontal and Medial Prefrontal Cortex to Probabilistic Reversal Learning.
      ,
      • Bari A.
      • Theobald D.E.
      • Caprioli D.
      • Mar A.C.
      • Aidoo-Micah A.
      • Dalley J.W.
      • et al.
      Serotonin Modulates Sensitivity to Reward and Negative Feedback in a Probabilistic Reversal Learning Task in Rats.
      ,
      • Ineichen C.
      • Sigrist H.
      • Spinelli S.
      • Lesch K.-P.
      • Sautter E.
      • Seifritz E.
      • et al.
      Establishing a probabilistic reversal learning test in mice: Evidence for the processes mediating reward-stay and punishment-shift behaviour and for their modulation by serotonin.
      ,
      • Waltz J.A.
      • Frank M.J.
      • Wiecki T.V.
      • Gold J.M.
      Altered probabilistic learning and response biases in schizophrenia: behavioral evidence and neurocomputational modeling.
      ). We observed decreased win-stay probability at 80:40, concordant with data from human studies using more complex contingency settings (
      • Budhani S.
      • Blair R.J.R.
      Response reversal and children with psychopathic tendencies: success is a function of salience of contingency change.
      ). These data demonstrate that complex PRL tasks can be translated for use in rodents.

      Amphetamine administration impairs punishment learning

      Striatal dopamine function is considered an important moderator of reversal learning (
      • Izquierdo A.
      • Brigman J.L.
      • Radke A.K.
      • Rudebeck P.H.
      • Holmes A.
      The neural basis of reversal learning: An updated perspective.
      ,
      • Kesby J.P.
      • Murray G.K.
      • Knolle F.
      Neural circuitry of salience and reward processing in psychosis.
      ,
      • Conn K.
      • Burne T.H.J.
      • Kesby J.P.
      Subcortical dopamine and cognition in schizophrenia: looking beyond psychosis in preclinical models.
      ,
      • Clatworthy P.L.
      • Lewis S.J.
      • Brichard L.
      • Hong Y.T.
      • Izquierdo D.
      • Clark L.
      • et al.
      Dopamine release in dissociable striatal subregions predicts the different effects of oral methylphenidate on reversal learning and spatial working memory.
      ,
      • Cools R.
      • Frank M.J.
      • Gibbs S.E.
      • Miyakawa A.
      • Jagust W.
      • D'Esposito M.
      Striatal dopamine predicts outcome-specific reversal learning and its sensitivity to dopaminergic drug administration.
      ,
      • Clarke H.F.
      • Hill G.J.
      • Robbins T.W.
      • Roberts A.C.
      Dopamine, But Not Serotonin, Regulates Reversal Learning in the Marmoset Caudate Nucleus.
      ). Increasing brain dopamine with amphetamine significantly reduced lose-shift probability and increased perseverative errors after a reversal. Acute amphetamine treatment has been shown to alter lose-shift performance in other protocols with misleading feedback (
      • Wong S.A.
      • Thapa R.
      • Badenhorst C.A.
      • Briggs A.R.
      • Sawada J.A.
      • Gruber A.J.
      Opposing effects of acute and chronic d-amphetamine on decision-making in rats.
      ) and dopamine depletion in the DMS increases lose-shift probability in rats (
      • Grospe G.M.
      • Baker P.M.
      • Ragozzino M.E.
      Cognitive Flexibility Deficits Following 6-OHDA Lesions of the Rat Dorsomedial Striatum.
      ). Moreover, increased perseverations and alterations in corticostriatal activity have been observed in stimulant abusers (
      • Ersche K.D.
      • Roiser J.P.
      • Abbott S.
      • Craig K.J.
      • Muller U.
      • Suckling J.
      • et al.
      Response perseveration in stimulant dependence is associated with striatal dysfunction and can be ameliorated by a D(2/3) receptor agonist.
      ). These outcomes complement the computational modelling parameters which suggest amphetamine impairs punishment learning and the response to loss (defined as a lack of reward in this protocol). Thus, amphetamine-treated mice place less emphasis on current unrewarded (losses) trials after a reversal, leading to a longer period selecting the now less optimal outcome. These findings align with previous studies highlighting the potential role of dopamine in learning, whereby dopamine depletion improves punishment-based reversal learning (
      • Mathar D.
      • Wilkinson L.
      • Holl A.K.
      • Neumann J.
      • Deserno L.
      • Villringer A.
      • et al.
      The role of dopamine in positive and negative prediction error utilization during incidental learning – Insights from Positron Emission Tomography, Parkinson's disease and Huntington's disease.
      ,
      • Robinson O.J.
      • Standing H.R.
      • DeVito E.E.
      • Cools R.
      • Sahakian B.J.
      Dopamine precursor depletion improves punishment prediction during reversal learning in healthy females but not males.
      ).
      Dopamine function often acts in an inverted-U-shaped response, with too little or too much impairing cognitive function (
      • Cools R.
      • D'Esposito M.
      Inverted-U-shaped dopamine actions on human working memory and cognitive control.
      ), meaning that the same dose of amphetamine can improve performance in some subjects, while impairing it in others (
      • Clatworthy P.L.
      • Lewis S.J.
      • Brichard L.
      • Hong Y.T.
      • Izquierdo D.
      • Clark L.
      • et al.
      Dopamine release in dissociable striatal subregions predicts the different effects of oral methylphenidate on reversal learning and spatial working memory.
      ,
      • Turner K.M.
      • Burne T.H.
      Improvement of attention with amphetamine in low- and high-performing rats.
      ,

      Kesby JP, Fields JA, Chang A, Coban H, Achim CL, Semenova S (2018): Effects of HIV-1 TAT protein and methamphetamine exposure on visual discrimination and executive function in mice. Behav Brain Res.73-79.

      ). Therefore, outcomes often differ between studies. For example, cocaine and D-amphetamine have been shown to decrease total reversals and win-stay probability; however lose-shift probability and perseverative errors remained consistent with those in saline-treated rats (
      • Verharen J.
      • De Jong H.
      • Roelofs T.
      • Huffels C.
      • Zessen R.
      • Luijendijk M.
      • et al.
      A neuronal mechanism underlying decision-making deficits during hyperdopaminergic states.
      ). Similarly, PRL performance and perseverative behavior in humans is not affected by administration of methylphenidate, a dopamine transporter blocker (

      Rostami Kandroodi M, Cook J, Swart J, Froböse M, Geurts D, Vahabie A, et al. (2020): Effects of methylphenidate on reversal learning depend on working memory capacity.

      ). These disparities may be due to differences in baseline dopamine function or competing regional effects after systemic administration.

      LO and DMS networks underlie reversal performance in mice

      Our cFos data demonstrate that amphetamine increases the activation of DMS cells and LO inputs to the DMS. Selective effects of amphetamine on DMS activity have also been previously observed (
      • Ren K.
      • Guo B.
      • Dai C.
      • Yao H.
      • Sun T.
      • Liu X.
      • et al.
      Striatal Distribution and Cytoarchitecture of Dopamine Receptor Subtype 1 and 2: Evidence from Double-Labeling Transgenic Mice.
      ), and the dorsal striatum may be more susceptible to stimulant-induced alterations in dopamine (
      • Kesby J.P.
      • Chang A.
      • Markou A.
      • Semenova S.
      Modeling human methamphetamine use patterns in mice: chronic and binge methamphetamine exposure, reward function and neurochemistry.
      ). The LO is also a core area associated with serial reversal learning, lose-shift probability and identifying changes in outcome contingencies (
      • Izquierdo A.
      Functional Heterogeneity within Rat Orbitofrontal Cortex in Reward Learning and Decision Making.
      ,

      Hervig ME, Fiddian L, Piilgaard L, Božič T, Blanco-Pozo M, Knudsen C, et al. (2020): Dissociable and Paradoxical Roles of Rat Medial and Lateral Orbitofrontal Cortex in Visual Serial Reversal Learning. Cerebral cortex (New York, NY : 1991). 30:1016-1029.

      ). The LO and DMS have overlapping roles, with studies in marmosets demonstrating that lesions of the caudate produce similar reversal learning phenotypes to those of the orbitofrontal cortex (
      • Clarke H.F.
      • Robbins T.W.
      • Roberts A.C.
      Lesions of the medial striatum in monkeys produce perseverative impairments during reversal learning similar to those produced by lesions of the orbitofrontal cortex.
      ). This suggests that LO inputs to the DMS may be responsible for downstream changes in activity and reversal learning outcomes. For example, our PCA suggests that increased activity in the LO inputs to the DMS, and the DMS, were both negatively associated with lose-shift probability. This aligns with the decreased lose-shift probability observed after amphetamine. However, these same measures were also positively associated with improved reversal performance and win-stay probability, indicating that LO to DMS signaling may drive multiple aspects of serial reversal learning. The balance of these competing roles, dose of amphetamine and behavioral context may underlie the disparity seen between studies (
      • Evenden J.L.
      • Robbins T.W.
      Increased response switching, perseveration and perseverative switching following d-amphetamine in the rat.
      ).

      The DMS regulates multiple aspects of reversal learning

      To clarify the role of the DMS in reversal learning, independent of alterations in activity from LO inputs, we used chemogenetic activation of the DMS. Acute activation did not impair reversal performance but there was a decrease in lose-shift probability at the 80:40 contingency. Specific decreases in lose-shift probability have been found at 80:40, but not 80:20, in those with early psychosis (

      Suetani S, Baker A, Garner K, Cosgrove P, Mackay-Sim M, Siskind D, et al. (2021): Impairments in goal-directed action and reversal learning in a proportion of individuals with psychosis: evidence for differential phenotypes in early and persistent psychosis. medRxiv.2021.2008.2031.21262937.

      ) which features increased dopamine function in the associative striatum (DMS equivalent in humans) (
      • Kesby J.P.
      • Murray G.K.
      • Knolle F.
      Neural circuitry of salience and reward processing in psychosis.
      ,
      • Kesby J.P.
      • Eyles D.W.
      • McGrath J.J.
      • Scott J.G.
      Dopamine, psychosis and schizophrenia: the widening gap between basic and clinical neuroscience.
      ). In addition, imaging studies in those at risk of developing psychosis (high levels of subthreshold psychotic symptoms) have observed altered caudate activation in response to unexpected feedback during PRL (

      Karcher NR, Hua JPY, Kerns JG (2019): Striatum-related functional activation during reward- versus punishment-based learning in psychosis risk. Neuropsychopharmacology : official publication of the American College of Neuropsychopharmacology. 44:1967-1974.

      ). The specific effects of amphetamine and DMS activation on lose-shift, but not win-stay, probability was unexpected. However, studies using cortical lesions in rats have suggested that value-related information (both reward and loss) is distributed across cortical areas allowing for redundancy and parallel processing (
      • Verharen J.P.H.
      • den Ouden H.E.M.
      • Adan R.A.H.
      • Vanderschuren L.
      Modulation of value-based decision making behavior by subregions of the rat prefrontal cortex.
      ). Potentially the DMS is more critical in navigating the response to loss than reward.
      We then used chronic activation to see if sustained impairments in DMS function could impair performance. Sustained DMS activation decreased lose-shift probability at 80:20 contingencies akin to amphetamine. Furthermore, reversal performance was heavily impacted at more difficult contingencies (80:40 and 70:30). This demonstrates that DMS dysfunction can replicate amphetamine-induced reversal learning phenotypes, and that DMS-induced alterations in lose-shift probability and reversal performance are distinct. For example, lose-shift probability was decreased at 80:20 but reversal performance was maintained. In contrast, lose-shift probability was maintained at 70:30 but reversal performance was significantly impaired. Although altered activation of the ventral striatum is often observed during reversal learning in those with schizophrenia (
      • Kesby J.P.
      • Murray G.K.
      • Knolle F.
      Neural circuitry of salience and reward processing in psychosis.
      ), there is evidence of altered caudate function. For example, decreased caudate activation in those with schizophrenia has been associated with deficits in probabilistic learning and after positive feedback (
      • Shepard P.D.
      • Holcomb H.H.
      • Gold J.M.
      The presence of absence: Habenular regulation of dopamine neurons and the encoding of negative outcomes.
      ,
      • Weickert T.W.
      • Goldberg T.E.
      • Callicott J.H.
      • Chen Q.
      • Apud J.A.
      • Das S.
      • et al.
      Neural correlates of probabilistic category learning in patients with schizophrenia.
      ). Together, these data extend former work on the DMS that highlights its complex role in the maintenance and reliable execution of a selected strategy (
      • Conn K.
      • Burne T.H.J.
      • Kesby J.P.
      Subcortical dopamine and cognition in schizophrenia: looking beyond psychosis in preclinical models.
      ,
      • Ragozzino M.E.
      The contribution of the medial prefrontal cortex, orbitofrontal cortex, and dorsomedial striatum to behavioral flexibility.
      ). Moreover, the DMS is important for evidence accumulation during learning, perhaps uniquely so in the brain (
      • Yartsev M.M.
      • Hanks T.D.
      • Yoon A.M.
      • Brody C.D.
      Causal contribution and dynamical encoding in the striatum during evidence accumulation.
      ,
      • Bolkan S.S.
      • Stone I.R.
      • Pinto L.
      • Ashwood Z.C.
      • Iravedra Garcia J.M.
      • Herman A.L.
      • et al.
      Opponent control of behavior by dorsomedial striatal pathways depends on task demands and internal state.
      ), which may become more important in more uncertain, complex environments. These multifaceted computations and roles support the possibility of multiple behavioral phenotypes due to DMS dysfunction, dependent on probabilistic uncertainty, and the level of training or chronicity. Imaging studies focused on how corticostriatal activation changes in response to probabilistic uncertainty may provide more clarity on the role of the DMS (or caudate) during PRL. Based on our data and that discussed here we would hypothesize that those with psychosis may show larger activation differences compared with healthy controls as uncertainty is increased.

      Conclusions

      Here we demonstrate that mice can perform many reversals in a single session of PRL, and that LO inputs to the DMS (and DMS function) are critical for navigating reversal learning. Furthermore, altering dopaminergic function globally and in the DMS can induce multiple reversal learning phenotypes, highlighting a complex interplay between contingency and DMS dopamine systems. These data suggest that the deficits observed in people with psychosis could be driven by subcortical dopaminergic dysfunction in the DMS, an effect which could occur independent of, or in addition to, cortical dysfunction in these individuals.

      Uncited reference

      R Core Team (2021): R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing.

      .

      Acknowledgments

      Imaging and analyses were performed at the Queensland Brain Institute's Advanced Microscopy Facility. This work was supported by an Advance Queensland Research Fellowship (AQRF04115-16RD1 to JPK), a University of Queensland Early Career Researcher Grant (JPK), the Brain & Behavior Research Foundation (Maltz Prize to JPK) and a National Health and Medical Research Council (NHMRC) Project Grant (GNT1139960 to JPK).

      References

      1. Everitt BJ, Parkinson JA, Olmstead MC, Arroyo M, Robledo P, Robbins TW (1999): Associative processes in addiction and reward - The role of amygdala-ventral striatal subsystems. In: McGinty JF, editor. Advancing from the Ventral Striatum to the Extended Amygdala: Implications for Neuropsychiatry and Drug Abuse: In Honor of Lennart Heimer. New York: New York Acad Sciences, pp 412-438.

        • Kehagia A.A.
        • Murray G.K.
        • Robbins T.W.
        Learning and cognitive flexibility: frontostriatal function and monoaminergic modulation.
        Current opinion in neurobiology. 2010; 20: 199-204
        • Murray G.K.
        • Cheng F.
        • Clark L.
        • Barnett J.H.
        • Blackwell A.D.
        • Fletcher P.C.
        • et al.
        Reinforcement and Reversal Learning in First-Episode Psychosis.
        Schizophr Bull. 2008; 34: 848-855
        • Waltz J.A.
        The neural underpinnings of cognitive flexibility and their disruption in psychotic illness.
        Neuroscience. 2016; : 203-217
        • Izquierdo A.
        • Brigman J.L.
        • Radke A.K.
        • Rudebeck P.H.
        • Holmes A.
        The neural basis of reversal learning: An updated perspective.
        Neuroscience. 2017; 345: 12-26
        • Keefe R.S.E.
        • Bilder R.M.
        • Harvey P.D.
        • Davis S.M.
        • Palmer B.W.
        • Gold J.M.
        • et al.
        Baseline Neurocognitive Deficits in the CATIE Schizophrenia Trial.
        Neuropsychopharmacology. 2006; 31: 2033-2046
        • Kesby J.P.
        • Murray G.K.
        • Knolle F.
        Neural circuitry of salience and reward processing in psychosis.
        Biological Psychiatry: GOS. 2021; (In press)https://doi.org/10.1016/j.bpsgos.2021.1012.1003
        • Culbreth A.J.
        • Gold J.M.
        • Cools R.
        • Barch D.M.
        Impaired Activation in Cognitive Control Regions Predicts Reversal Learning in Schizophrenia.
        Schizophr Bull. 2016; 42: 484-493
        • Waltz J.A.
        • Gold J.M.
        Probabilistic reversal learning impairments in schizophrenia: Further evidence of orbitofrontal dysfunction.
        Schizophr Res. 2007; 93: 296-303
        • Suetani S.
        • Baker A.
        • Garner K.
        • Cosgrove P.
        • Mackay-Sim M.
        • Siskind D.
        • et al.
        Impairments in goal-directed action and reversal learning in a proportion of individuals with psychosis.
        Cognitive, Affective, & Behavioral Neuroscience. 2022;
        • Schlagenhauf F.
        • Huys Q.J.
        • Deserno L.
        • Rapp M.A.
        • Beck A.
        • Heinze H.J.
        • et al.
        Striatal dysfunction during reversal learning in unmedicated schizophrenia patients.
        Neuroimage. 2014; 89: 171-180
        • Weiler J.A.
        • Bellebaum C.
        • Brüne M.
        • Juckel G.
        • Daum I.
        Impairment of probabilistic reward-based learning in schizophrenia.
        Neuropsychology. 2009; 23: 571-580
        • Murray G.K.
        • Cheng F.
        • Clark L.
        • Barnett J.H.
        • Blackwell A.D.
        • Fletcher P.C.
        • et al.
        Reinforcement and reversal learning in first-episode psychosis.
        Schizophr Bull. 2008; 34: 848-855
        • Waltz J.A.
        • Kasanova Z.
        • Ross T.J.
        • Salmeron B.J.
        • McMahon R.P.
        • Gold J.M.
        • et al.
        The roles of reward, default, and executive control networks in set-shifting impairments in schizophrenia.
        PLoS One. 2013; 8e57257
        • Conn K.
        • Burne T.H.J.
        • Kesby J.P.
        Subcortical dopamine and cognition in schizophrenia: looking beyond psychosis in preclinical models.
        Front Neurosci. 2020; 14: 542
        • Kesby J.P.
        • Eyles D.W.
        • McGrath J.J.
        • Scott J.G.
        Dopamine, psychosis and schizophrenia: the widening gap between basic and clinical neuroscience.
        Translational psychiatry. 2018; 8: 30
        • Clatworthy P.L.
        • Lewis S.J.
        • Brichard L.
        • Hong Y.T.
        • Izquierdo D.
        • Clark L.
        • et al.
        Dopamine release in dissociable striatal subregions predicts the different effects of oral methylphenidate on reversal learning and spatial working memory.
        J Neurosci. 2009; 29: 4690-4696
        • Dalton G.L.
        • Wang N.Y.
        • Phillips A.G.
        • Floresco S.B.
        Multifaceted Contributions by Different Regions of the Orbitofrontal and Medial Prefrontal Cortex to Probabilistic Reversal Learning.
        The Journal of Neuroscience. 2016; 36: 1996-2006
        • Robinson O.J.
        • Frank M.J.
        • Sahakian B.J.
        • Cools R.
        Dissociable responses to punishment in distinct striatal regions during reversal learning.
        NeuroImage. 2010; 51: 1459-1467
        • Ragozzino M.E.
        The contribution of the medial prefrontal cortex, orbitofrontal cortex, and dorsomedial striatum to behavioral flexibility.
        Annals of the New York Academy of Sciences. 2007; 1121: 355-375
        • Clarke H.F.
        • Robbins T.W.
        • Roberts A.C.
        Lesions of the medial striatum in monkeys produce perseverative impairments during reversal learning similar to those produced by lesions of the orbitofrontal cortex.
        J Neurosci. 2008; 28: 10972-10982
        • Clarke H.F.
        • Cardinal R.N.
        • Rygula R.
        • Hong Y.T.
        • Fryer T.D.
        • Sawiak S.J.
        • et al.
        Orbitofrontal dopamine depletion upregulates caudate dopamine and alters behavior via changes in reinforcement sensitivity.
        J Neurosci. 2014; 34: 7663-7676
        • Izquierdo A.
        Functional Heterogeneity within Rat Orbitofrontal Cortex in Reward Learning and Decision Making.
        J Neurosci. 2017; 37: 10529-10540
        • Milienne-Petiot M.
        • Kesby J.P.
        • Graves M.
        • van Enkhuizen J.
        • Semenova S.
        • Minassian A.
        • et al.
        The effects of reduced dopamine transporter function and chronic lithium on motivation, probabilistic learning, and neurochemistry in mice: Modeling bipolar mania.
        Neuropharmacology. 2017; 113: 260-270
        • Metha J.A.
        • Brian M.L.
        • Oberrauch S.
        • Barnes S.A.
        • Featherby T.J.
        • Bossaerts P.
        • et al.
        Separating Probability and Reversal Learning in a Novel Probabilistic Reversal Learning Task for Mice.
        Frontiers in behavioral neuroscience. 2020; 13
      2. Paxinos G, Franklin K (2001): The Mouse Brain in Stereotaxic Coordinates. 2nd ed. San Diego: Academic Press.

        • Ahn W.Y.
        • Haines N.
        • Zhang L.
        Revealing Neurocomputational Mechanisms of Reinforcement Learning and Decision-Making With the hBayesDM Package.
        Computational psychiatry (Cambridge, Mass). 2017; 1: 24-57
        • den Ouden H.E.
        • Daw N.D.
        • Fernandez G.
        • Elshout J.A.
        • Rijpkema M.
        • Hoogman M.
        • et al.
        Dissociable effects of dopamine and serotonin on reversal learning.
        Neuron. 2013; 80: 1090-1100
        • Schneider C.A.
        • Rasband W.S.
        • Eliceiri K.W.
        NIH Image to ImageJ: 25 years of image analysis.
        Nature Methods. 2012; 9: 671-675
        • Stringer C.
        • Wang T.
        • Michaelos M.
        • Pachitariu M.
        Cellpose: a generalist algorithm for cellular segmentation.
        Nature Methods. 2021; 18: 100-106
        • Stirling D.R.
        • Swain-Bowden M.J.
        • Lucas A.M.
        • Carpenter A.E.
        • Cimini B.A.
        • Goodman A.
        CellProfiler 4: improvements in speed, utility and usability.
        BMC Bioinformatics. 2021; 22: 433
      3. R Core Team (2021): R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing.

        • Waltmann M.
        • Schlagenhauf F.
        • Deserno L.
        Sufficient reliability of the behavioral and computational readouts of a probabilistic reversal learning task.
        Behavior Research Methods. 2022;
      4. Angrist B (1994): Amphetamine psychosis: clinical variations of the syndrome. In: Cho AK, Segal DS, editors. Amphetamine and its analogues. San Diego: Academic Press, pp 387-414.

        • Krystal J.H.
        • Perry Jr., E.B.
        • Gueorguieva R.
        • Belger A.
        • Madonick S.H.
        • Abi-Dargham A.
        • et al.
        Comparative and interactive human psychopharmacologic effects of ketamine and amphetamine: implications for glutamatergic and dopaminergic model psychoses and cognitive function.
        Arch Gen Psychiatry. 2005; 62: 985-994
        • Robinson T.E.
        • Becker J.B.
        Enduring changes in brain and behavior produced by chronic amphetamine administration: a review and evaluation of animal models of amphetamine psychosis.
        Brain Res. 1986; 396: 157-198
        • Abi-Dargham A.
        • van de Giessen E.
        • Slifstein M.
        • Kegeles L.S.
        • Laruelle M.
        Baseline and Amphetamine-Stimulated Dopamine Activity Are Related in Drug-Naive Schizophrenic Subjects.
        Biol Psychiatry. 2009; 65: 1091-1093
        • Lipska B.K.
        • Jaskiw G.E.
        • Weinberger D.R.
        Postpubertal Emergence of Hyperresponsiveness to Stress and to Amphetamine after Neonatal Excitotoxic Hippocampal Damage - a Potential Animal-Model of Schizophrenia.
        Neuropsychopharmacology. 1993; 9: 67-75
        • Petty A.
        • Cui X.
        • Ali A.
        • Du Z.
        • Srivastav S.
        • Kesby J.P.
        • et al.
        Positive symptom phenotypes appear progressively in "EDiPS", a new animal model of the schizophrenia prodrome.
        Sci Rep. 2021; 11: 4294
        • Laruelle M.
        • Abi-Dargham A.
        • Gil R.
        • Kegeles L.
        • Innis R.
        Increased dopamine transmission in schizophrenia: Relationship to illness phases.
        Biol Psychiat. 1999; 46: 56-72
        • Heilbronner S.R.
        • Rodriguez-Romaguera J.
        • Quirk G.J.
        • Groenewegen H.J.
        • Haber S.N.
        Circuit-Based Corticostriatal Homologies Between Rat and Primate.
        Biol Psychiatry. 2016; 80: 509-521
        • Mailly P.
        • Aliane V.
        • Groenewegen H.J.
        • Haber S.N.
        • Deniau J.M.
        The rat prefrontostriatal system analyzed in 3D: evidence for multiple interacting functional units.
        J Neurosci. 2013; 33: 5718-5727
        • Hunnicutt B.J.
        • Jongbloets B.C.
        • Birdsong W.T.
        • Gertz K.J.
        • Zhong H.
        • Mao T.
        A comprehensive excitatory input map of the striatum reveals novel functional organization.
        eLife. 2016; 5
        • Ren K.
        • Guo B.
        • Dai C.
        • Yao H.
        • Sun T.
        • Liu X.
        • et al.
        Striatal Distribution and Cytoarchitecture of Dopamine Receptor Subtype 1 and 2: Evidence from Double-Labeling Transgenic Mice.
        Frontiers in Neural Circuits. 2017; 11
        • Ventura R.
        • Alcaro A.
        • Cabib S.
        • Conversi D.
        • Mandolesi L.
        • Puglisi-Allegra S.
        Dopamine in the medial prefrontal cortex controls genotype-dependent effects of amphetamine on mesoaccumbens dopamine release and locomotion.
        Neuropsychopharmacology. 2004; 29: 72-80
        • Grant A.
        • Hoops D.
        • Labelle-Dumais C.
        • Prévost M.
        • Rajabi H.
        • Kolb B.
        • et al.
        Netrin-1 receptor-deficient mice show enhanced mesocortical dopamine transmission and blunted behavioural responses to amphetamine.
        Eur J Neurosci. 2007; 26: 3215-3228
        • Cools R.
        • Frank M.J.
        • Gibbs S.E.
        • Miyakawa A.
        • Jagust W.
        • D'Esposito M.
        Striatal dopamine predicts outcome-specific reversal learning and its sensitivity to dopaminergic drug administration.
        J Neurosci. 2009; 29: 1538-1543
        • Urban D.J.
        • Zhu H.
        • Marcinkiewcz C.A.
        • Michaelides M.
        • Oshibuchi H.
        • Rhea D.
        • et al.
        Elucidation of The Behavioral Program and Neuronal Network Encoded by Dorsal Raphe Serotonergic Neurons.
        Neuropsychopharmacology. 2016; 41: 1404-1415
      5. Suetani S, Baker A, Garner K, Cosgrove P, Mackay-Sim M, Siskind D, et al. (2021): Impairments in goal-directed action and reversal learning in a proportion of individuals with psychosis: evidence for differential phenotypes in early and persistent psychosis. medRxiv.2021.2008.2031.21262937.

        • Bari A.
        • Theobald D.E.
        • Caprioli D.
        • Mar A.C.
        • Aidoo-Micah A.
        • Dalley J.W.
        • et al.
        Serotonin Modulates Sensitivity to Reward and Negative Feedback in a Probabilistic Reversal Learning Task in Rats.
        Neuropsychopharmacology. 2010; 35: 1290-1301
        • Ineichen C.
        • Sigrist H.
        • Spinelli S.
        • Lesch K.-P.
        • Sautter E.
        • Seifritz E.
        • et al.
        Establishing a probabilistic reversal learning test in mice: Evidence for the processes mediating reward-stay and punishment-shift behaviour and for their modulation by serotonin.
        Neuropharmacology. 2012; 63: 1012-1021
        • Waltz J.A.
        • Frank M.J.
        • Wiecki T.V.
        • Gold J.M.
        Altered probabilistic learning and response biases in schizophrenia: behavioral evidence and neurocomputational modeling.
        Neuropsychology. 2011; 25: 86-97
        • Budhani S.
        • Blair R.J.R.
        Response reversal and children with psychopathic tendencies: success is a function of salience of contingency change.
        J Child Psychol Psychiatry. 2005; 46: 972-981
        • Clarke H.F.
        • Hill G.J.
        • Robbins T.W.
        • Roberts A.C.
        Dopamine, But Not Serotonin, Regulates Reversal Learning in the Marmoset Caudate Nucleus.
        J Neurosci. 2011; 31: 4290-4297
        • Wong S.A.
        • Thapa R.
        • Badenhorst C.A.
        • Briggs A.R.
        • Sawada J.A.
        • Gruber A.J.
        Opposing effects of acute and chronic d-amphetamine on decision-making in rats.
        Neuroscience. 2017; 345: 218-228
        • Grospe G.M.
        • Baker P.M.
        • Ragozzino M.E.
        Cognitive Flexibility Deficits Following 6-OHDA Lesions of the Rat Dorsomedial Striatum.
        Neuroscience. 2018; 374: 80-90
        • Ersche K.D.
        • Roiser J.P.
        • Abbott S.
        • Craig K.J.
        • Muller U.
        • Suckling J.
        • et al.
        Response perseveration in stimulant dependence is associated with striatal dysfunction and can be ameliorated by a D(2/3) receptor agonist.
        Biol Psychiatry. 2011; 70: 754-762
        • Mathar D.
        • Wilkinson L.
        • Holl A.K.
        • Neumann J.
        • Deserno L.
        • Villringer A.
        • et al.
        The role of dopamine in positive and negative prediction error utilization during incidental learning – Insights from Positron Emission Tomography, Parkinson's disease and Huntington's disease.
        Cortex. 2017; 90: 149-162
        • Robinson O.J.
        • Standing H.R.
        • DeVito E.E.
        • Cools R.
        • Sahakian B.J.
        Dopamine precursor depletion improves punishment prediction during reversal learning in healthy females but not males.
        Psychopharmacology (Berl). 2010; 211: 187-195
        • Cools R.
        • D'Esposito M.
        Inverted-U-shaped dopamine actions on human working memory and cognitive control.
        Biol Psychiatry. 2011; 69: e113-125
        • Turner K.M.
        • Burne T.H.
        Improvement of attention with amphetamine in low- and high-performing rats.
        Psychopharmacology (Berl). 2016; 233: 3383-3394
      6. Kesby JP, Fields JA, Chang A, Coban H, Achim CL, Semenova S (2018): Effects of HIV-1 TAT protein and methamphetamine exposure on visual discrimination and executive function in mice. Behav Brain Res.73-79.

        • Verharen J.
        • De Jong H.
        • Roelofs T.
        • Huffels C.
        • Zessen R.
        • Luijendijk M.
        • et al.
        A neuronal mechanism underlying decision-making deficits during hyperdopaminergic states.
        Nature communications. 2018; 9
      7. Rostami Kandroodi M, Cook J, Swart J, Froböse M, Geurts D, Vahabie A, et al. (2020): Effects of methylphenidate on reversal learning depend on working memory capacity.

        • Kesby J.P.
        • Chang A.
        • Markou A.
        • Semenova S.
        Modeling human methamphetamine use patterns in mice: chronic and binge methamphetamine exposure, reward function and neurochemistry.
        Addict Biol. 2018; 23: 206-218
      8. Hervig ME, Fiddian L, Piilgaard L, Božič T, Blanco-Pozo M, Knudsen C, et al. (2020): Dissociable and Paradoxical Roles of Rat Medial and Lateral Orbitofrontal Cortex in Visual Serial Reversal Learning. Cerebral cortex (New York, NY : 1991). 30:1016-1029.

        • Evenden J.L.
        • Robbins T.W.
        Increased response switching, perseveration and perseverative switching following d-amphetamine in the rat.
        Psychopharmacology (Berl). 1983; 80: 67-73
      9. Karcher NR, Hua JPY, Kerns JG (2019): Striatum-related functional activation during reward- versus punishment-based learning in psychosis risk. Neuropsychopharmacology : official publication of the American College of Neuropsychopharmacology. 44:1967-1974.

        • Verharen J.P.H.
        • den Ouden H.E.M.
        • Adan R.A.H.
        • Vanderschuren L.
        Modulation of value-based decision making behavior by subregions of the rat prefrontal cortex.
        Psychopharmacology (Berl). 2020; 237: 1267-1280
        • Shepard P.D.
        • Holcomb H.H.
        • Gold J.M.
        The presence of absence: Habenular regulation of dopamine neurons and the encoding of negative outcomes.
        Schizophr Bull. 2006; 32: 417-421
        • Weickert T.W.
        • Goldberg T.E.
        • Callicott J.H.
        • Chen Q.
        • Apud J.A.
        • Das S.
        • et al.
        Neural correlates of probabilistic category learning in patients with schizophrenia.
        J Neurosci. 2009; 29: 1244-1254
        • Yartsev M.M.
        • Hanks T.D.
        • Yoon A.M.
        • Brody C.D.
        Causal contribution and dynamical encoding in the striatum during evidence accumulation.
        eLife. 2018; 7
        • Bolkan S.S.
        • Stone I.R.
        • Pinto L.
        • Ashwood Z.C.
        • Iravedra Garcia J.M.
        • Herman A.L.
        • et al.
        Opponent control of behavior by dorsomedial striatal pathways depends on task demands and internal state.
        Nat Neurosci. 2022; 25: 345-357