Advertisement

Neural Circuitry of Salience and Reward Processing in Psychosis

  • James P. Kesby
    Correspondence
    Address correspondence to James Kesby, Ph.D.
    Affiliations
    Queensland Brain Institute, The University of Queensland, Brisbane, Queensland, Australia

    QIMR Berghofer Medical Research Institute, Brisbane, Queensland, Australia
    Search for articles by this author
  • Graham K. Murray
    Affiliations
    Institute for Molecular Bioscience, The University of Queensland, Brisbane, Queensland, Australia

    Department of Psychiatry, University of Cambridge, Cambridge, United Kingdom

    Cambridgeshire and Peterborough NHS Foundation Trust, Cambridge, United Kingdom
    Search for articles by this author
  • Franziska Knolle
    Correspondence
    Franziska Knolle, Ph.D.
    Affiliations
    Department of Psychiatry, University of Cambridge, Cambridge, United Kingdom

    Department of Diagnostic and Interventional Neuroradiology, School of Medicine, Technical University of Munich, Munich, Germany
    Search for articles by this author
Open AccessPublished:December 10, 2021DOI:https://doi.org/10.1016/j.bpsgos.2021.12.003

      Abstract

      The processing of salient and rewarding stimuli is integral to engaging our attention, stimulating anticipation for future events, and driving goal-directed behaviors. Widespread impairments in these processes are observed in psychosis, which may be associated with worse functional outcomes or mechanistically linked to the development of symptoms. Here, we summarize the current knowledge of behavioral and functional neuroimaging in salience, prediction error, and reward. Although each is a specific process, they are situated in multiple feedback and feedforward systems integral to decision making and cognition more generally. We argue that the origin of salience and reward processing dysfunctions may be centered in the subcortex during the earliest stages of psychosis, with cortical abnormalities being initially more spared but becoming more prominent in established psychotic illness/schizophrenia. The neural circuits underpinning salience and reward processing may provide targets for delaying or preventing progressive behavioral and neurobiological decline.

      Keywords

      SEE COMMENTARY ON PAGE 6
      Psychoses of the schizophrenia spectrum are characterized by positive symptoms, such as hallucinations and delusions, which are dominant during the psychotic stages of the disease; negative symptoms, such as lack of motivation and emotional blunting; and cognitive symptoms, such as memory dysfunctions (
      • McCutcheon R.A.
      • Reis Marques T.
      • Howes O.D.
      Schizophrenia-An overview.
      ). Although the underlying mechanisms leading to this complex set of symptoms are not fully understood, one of the most robust findings in psychosis and schizophrenia and even in the prodromal stages of the disease [e.g., (
      • Howes O.D.
      • Egerton A.
      • Allan V.
      • McGuire P.
      • Stokes P.
      • Kapur S.
      Mechanisms underlying psychosis and antipsychotic treatment response in schizophrenia: Insights from PET and SPECT imaging.
      )] is the elevation of striatal dopamine [e.g., (
      • Howes O.D.
      • Montgomery A.J.
      • Asselin M.C.
      • Murray R.M.
      • Valli I.
      • Tabraham P.
      • et al.
      Elevated striatal dopamine function linked to prodromal signs of schizophrenia.
      )]. Dopamine is therefore the target of most antipsychotic pharmacological interventions (
      • McCutcheon R.A.
      • Reis Marques T.
      • Howes O.D.
      Schizophrenia-An overview.
      ), greatly reducing positive symptoms (
      • Kapur S.
      • Mamo D.
      Half a century of antipsychotics and still a central role for dopamine D2 receptors.
      ). Unfortunately, however, approximately 30% of patients with schizophrenia classify as treatment resistant to dopamine D2 receptor antagonists and have worse long-term functional disability, with more severe positive, negative, and cognitive symptoms [e.g., (
      • Nucifora Jr., F.C.
      • Woznica E.
      • Lee B.J.
      • Cascella N.
      • Sawa A.
      Treatment resistant schizophrenia: Clinical, biological, and therapeutic perspectives.
      )]. Glutamatergic systems may be more relevant than dopamine for the pathogenesis of positive psychotic symptoms in these individuals (
      • Demjaha A.
      • Egerton A.
      • Murray R.M.
      • Kapur S.
      • Howes O.D.
      • Stone J.M.
      • McGuire P.K.
      Antipsychotic treatment resistance in schizophrenia associated with elevated glutamate levels but normal dopamine function.
      ). Similarly, links between negative and cognitive symptoms and dysregulation of the glutamatergic system have also been observed [e.g., (
      • Uno Y.
      • Coyle J.T.
      Glutamate hypothesis in schizophrenia.
      )]. However, pharmacological interventions targeting the glutamatergic system have been less successful (
      • McCutcheon R.A.
      • Reis Marques T.
      • Howes O.D.
      Schizophrenia-An overview.
      ). Given the strong associations between dopamine, salience/reward, and negative symptoms, this review is focused largely on subcortical dopamine systems and their associated corticostriatal networks.
      In this article, we examine evidence for dysfunction of the neural circuitry subserving reward and salience processing and their links to dopaminergic dysregulation in the psychosis spectrum (Table 1). These dysfunctions are potentially highly relevant for understanding the core aspects of schizophrenia and related symptoms, including positive symptoms and negative symptoms. A brief introduction to the psychological processes included in salience, prediction error, and reward processing is provided in Figure 1. We endeavored to examine the extent to which neural dysfunctions have been linked to clinical symptoms, cognitive impairments, or functional disability because not all neural abnormalities may have deleterious consequences. Much of the work in this field is cross-sectional; however, we have endeavored to discuss jointly all stages of illness to highlight the relevant similarities and differences as much as possible.
      Table 1Psychosis Spectrum
      Disease StageDescription
      At RiskBy at risk, we mainly refer to people who are at increased risk of psychotic illness due to being help-seeking patients presenting with mild (subthreshold) clinical symptoms, especially subthreshold positive psychotic symptoms, such as suspicions or hallucinations without delusional interpretations. Such individuals are sometimes termed ultra-high-risk, clinical high-risk, at-risk mental state, or prodromal psychosis, with several (slightly differing) operational criteria available to categorize people in such states (
      • Thompson A.
      • Marwaha S.
      • Broome M.R.
      At-risk mental state for psychosis: Identification and current treatment approaches.
      ). We note that although such groups are especially at (relatively) high risk of psychosis, they also are at risk for other adverse psychiatric outcomes (
      • McGorry P.D.
      • Hartmann J.A.
      • Spooner R.
      • Nelson B.
      Beyond the “at risk mental state” concept: Transitioning to transdiagnostic psychiatry.
      ).
      Early PsychosisBy early psychosis, we refer to the early stages after the onset of established psychotic illness, such as first-episode psychosis and first-episode schizophrenia. While some studies only include patients who meet the diagnostic criteria for schizophrenia, a number of research studies include a broader mixture of patients with first-episode psychosis. These are people presenting with psychotic illness for the first time, many with nonaffective schizophrenia spectrum psychosis, and others with affective psychosis such as bipolar disorder or depressive psychosis.
      Chronic Psychosis/SchizophreniaIn general, we use the term chronic psychosis/schizophrenia for referring to patients who have been unwell beyond the early stages of illness/first 5 years of psychotic illness. Where studies have specified a minimum duration of illness, we use the term chronic schizophrenia, but we note that some studies include a mixture of patients with schizophrenia at different stages of illness.
      Figure thumbnail gr1
      Figure 1Conceptual interactions between external events (cyan), salience and prediction error (red), and reward processing (purple) in decision making. An external event (e.g., action outcome) will be perceived and attended to when it overcomes a certain level of salience (referred to as a salience threshold). The input is then integrated into associative learning networks to inform decision-making processes. Computation of potential rewards available (valuation) and the effort required for each (effort-cost trade-off) are then used to identify the optimal choice and whether or not it is acted upon. In this example, the outcome is rewarding (external feedback as action outcome), which is then compared with our prior expectations. Our prior expectation is associated with how accurate our valuation of the outcome is. In cases where the outcome matches the expectation (green tick), the associations are reinforced. In cases where there is a mismatch between the outcome and our expectation, a prediction error signal is generated. The prediction error is used to update our understanding of input-output relationships. Prediction error information (magnitude, precision, and so on) is then used to update our beliefs and goals associated with the initial stimuli (or action required). This updates our salience threshold so that when we encounter this same event in the future (n + 1), it is more likely to be considered salient. Reward processing (purple) affects multiple stages in this process to subsequently increase attention (reward anticipation), drive associative learning (reward learning), and govern our motivation to work toward a future goal. Here, we focus on a rewarding outcome; however, perceptual and attentional processes of sensory stimuli work similarly, causing belief updating and salience threshold updating via prediction errors.

      Salience Processing in Psychosis

      Salience is a property that characterizes the importance of a stimulus and ultimately attracts attention to drive cognition and behavior. Salience is a multifaceted concept (
      • Winton-Brown T.T.
      • Fusar-Poli P.
      • Ungless M.A.
      • Howes O.D.
      Dopaminergic basis of salience dysregulation in psychosis.
      ) including different dimensions, such as reward or novelty (Box 1). Abnormal salience processing following dysregulation of the dopaminergic system has been linked to the formation and maintenance of positive and negative symptoms (
      • Abboud R.
      • Roiser J.P.
      • Khalifeh H.
      • Ali S.
      • Harrison I.
      • Killaspy H.T.
      • Joyce E.M.
      Are persistent delusions in schizophrenia associated with aberrant salience?.
      ,
      • Murray G.K.
      • Corlett P.R.
      • Fletcher P.C.
      The neural underpinnings of associative learning in health and psychosis: How can performance be preserved when brain responses are abnormal?.
      ,
      • Corlett P.R.
      • Murray G.K.
      • Honey G.D.
      • Aitken M.R.F.
      • Shanks D.R.
      • Robbins T.W.
      • et al.
      Disrupted prediction-error signal in psychosis: Evidence for an associative account of delusions.
      ,
      • Maia T.V.
      • Frank M.J.
      An integrative perspective on the role of dopamine in schizophrenia.
      ) and is referred to as the aberrant salience hypothesis of psychosis (
      • Winton-Brown T.T.
      • Fusar-Poli P.
      • Ungless M.A.
      • Howes O.D.
      Dopaminergic basis of salience dysregulation in psychosis.
      ,
      • Howes O.D.
      • Nour M.M.
      Dopamine and the aberrant salience hypothesis of schizophrenia.
      ,
      • Kapur S.
      Psychosis as a state of aberrant salience: A framework linking biology, phenomenology, and pharmacology in schizophrenia.
      ,
      • Heinz A.
      Dopaminergic dysfunction in alcoholism and schizophrenia—Psychopathological and behavioral correlates.
      ). According to the aberrant salience hypothesis, elevated levels of dopamine in psychosis [e.g., (
      • Fusar-Poli P.
      • Meyer-Lindenberg A.
      Striatal presynaptic dopamine in schizophrenia, part II: Meta-analysis of [(18)F/(11)C]-DOPA PET studies.
      )] create neurobiological noise, which is misinterpreted as meaningfulness and may lead to the attribution of salience to the otherwise unimportant, ordinary experiences that incidentally co-occur with this experience. The interpretation of these falsely judged–important stimuli may lead to the formation of hallucinations and delusions.
      Salience
      The world around us is highly complex and produces constant noisy and ambiguous sensory input to our brain. The biggest challenge for our brain is to rapidly and efficiently identify important stimuli and to process them effectively. One efficient way that the brain applies is to evaluate the saliency of incoming sensations and prioritize them accordingly considering the context. Imagine, for example, the change of the traffic lights from green to red; the change in the physical qualities of the visual input is highly informative and important to adapt our behavior and decisions. This example shows that there is a strong interaction between stimulus-driven processing and goals or belief of the individual to determine the saliency of incoming information.
      We can differentiate between different forms of saliency (
      • Winton-Brown T.
      • Schmidt A.
      • Roiser J.P.
      • Howes O.D.
      • Egerton A.
      • Fusar-Poli P.
      • et al.
      Altered activation and connectivity in a hippocampal-basal ganglia-midbrain circuit during salience processing in subjects at ultra high risk for psychosis.
      ), the first distinction being between incentive or motivational salience and nonmotivational salience. Incentive salience describes the desire to obtain a reward by increasing attention and motivational drive (see the sections on Prediction Error Signaling in Psychosis and Reward Processing in Psychosis for more information). Tasks used to investigate motivational salience use mainly monetary rewards. Nonmotivational salience, which is usually studied in visual/auditory oddball paradigms or when investigating the processing of irrelevant stimuli in monetary reward paradigms, can be distinguished in novelty salience and surprisal (
      • Barto A.
      • Mirolli M.
      • Baldassarre G.
      Novelty or surprise?.
      ), which are essential drivers of intrinsic motivation (not reward related) and attention. Surprise is characterized by a change in condition through the comparison of the expected to the actually perceived (e.g., the physical change of a stimulus or emotional change) and is strongly linked to an unsigned (i.e., without valence indication, worse or better than expected) prediction error (
      • Fouragnan E.
      • Retzler C.
      • Philiastides M.G.
      Separate neural representations of prediction error valence and surprise: Evidence from an fMRI meta-analysis.
      ). For novelty, however, the concept is less concrete. Generally, novelty refers to a sensory input never encountered before (complete novelty) or not encountered for some time (short-term/long-term novelty). See Barto et al. (
      • Barto A.
      • Mirolli M.
      • Baldassarre G.
      Novelty or surprise?.
      ) for details. Once some sensory input is being evaluated as salient, this is ultimately a driver for learning and behavior (
      • Rumbaugh D.M.
      • King J.E.
      • Beran M.J.
      • Washburn D.A.
      • Gould K.L.
      A salience theory of learning and behavior: With perspectives on neurobiology and cognition.
      ).
      When experimentally investigating salience processing, two different aspects might be explored: the processing of salient, such as emotional, novel, or motivational stimuli, or the processing of irrelevant, uninformative, neutral stimuli. In schizophrenia, deficits in the first aspect conceptually relate to the development and preservation of negative symptoms (
      • Maia T.V.
      • Frank M.J.
      An integrative perspective on the role of dopamine in schizophrenia.
      ,
      • Corlett P.R.
      • Honey G.D.
      • Fletcher P.C.
      From prediction error to psychosis: Ketamine as a pharmacological model of delusions.
      ); the latter suggests the overweighting of irrelevant stimuli, which is linked to the emergence of positive symptoms. The current literature provides evidence for aberrant salience processing across both aspects in early psychosis and chronic schizophrenia; however, links to symptomatology are less concise.
      At the same time, relevant stimuli, such as a reward prediction error or an emotional stimulus, fail to be processed appropriately, leading to a blunted response, potentially explaining negative symptoms (
      • Maia T.V.
      • Frank M.J.
      An integrative perspective on the role of dopamine in schizophrenia.
      ,
      • Corlett P.R.
      • Honey G.D.
      • Fletcher P.C.
      From prediction error to psychosis: Ketamine as a pharmacological model of delusions.
      ). The human and animal literature describes the critical role of dopamine in reward prediction error processing (
      • Schultz W.
      Predictive reward signal of dopamine neurons.
      ,
      • Schultz W.
      Dopamine reward prediction-error signalling: A two-component response.
      ). Reward prediction errors are intrinsically salient (see Prediction Error Signaling in Psychosis and Reward Processing in Psychosis). The firing of dopamine neurons, however, is not exclusive to reward prediction error but has been reported in response to nonrewarding unexpected events, such as aversive or alerting (
      • Corlett P.R.
      • Murray G.K.
      • Honey G.D.
      • Aitken M.R.F.
      • Shanks D.R.
      • Robbins T.W.
      • et al.
      Disrupted prediction-error signal in psychosis: Evidence for an associative account of delusions.
      ,
      • Horvitz J.C.
      Mesolimbocortical and nigrostriatal dopamine responses to salient non− reward events.
      ), as well as novel events (
      • Bunzeck N.
      • Doeller C.F.
      • Dolan R.J.
      • Duzel E.
      Contextual interaction between novelty and reward processing within the mesolimbic system.
      ), surprising events (
      • Haarsma J.
      • Fletcher P.C.
      • Griffin J.D.
      • Taverne H.J.
      • Ziauddeen H.
      • Spencer T.J.
      • et al.
      Precision weighting of cortical unsigned prediction error signals benefits learning, is mediated by dopamine, and is impaired in psychosis [published correction appears in Mol Psychiatry 2021; 26:5334].
      ), or physical change (
      • Valdés-Baizabal C.
      • Carbajal G.V.
      • Pérez-González D.
      • Malmierca M.S.
      Dopamine modulates subcortical responses to surprising sounds [published correction appears in PLoS Biol 2020; 18:e3000984].
      ). Therefore, dopamine release, at least in some contexts, may reflect general salience (
      • Bromberg-Martin E.S.
      • Matsumoto M.
      • Hikosaka O.
      Dopamine in motivational control: Rewarding, aversive, and alerting.
      ).

      Aberrant Salience as Altered Processing of Irrelevant Information

      Several studies investigated the processing of neutral or uninformative stimuli using different methods and exploring different stages of the disease and the link to symptoms; while theoretically clear, the experimental results show inconsistencies. Roiser et al. (
      • Roiser J.P.
      • Stephan K.E.
      • Den Ouden H.E.M.
      • Barnes T.R.E.
      • Friston K.J.
      • Joyce E.M.
      Do patients with schizophrenia exhibit aberrant salience?.
      ) adapted a monetary reinforcement learning task (salience attribution task) to investigate the salience assigned to irrelevant stimuli. Their study revealed that schizophrenia patients with delusions showed higher levels of aberrant salience to irrelevant stimuli than patients without delusions (
      • Roiser J.P.
      • Stephan K.E.
      • Den Ouden H.E.M.
      • Barnes T.R.E.
      • Friston K.J.
      • Joyce E.M.
      Do patients with schizophrenia exhibit aberrant salience?.
      ); however, aberrant salience was correlated with negative symptoms. Applying the same task to at-risk individuals, aberrant salience attribution to irrelevant stimuli was associated with the severity of positive symptoms (
      • Roiser J.P.
      • Howes O.D.
      • Chaddock C.A.
      • Joyce E.M.
      • McGuire P.
      Neural and behavioral correlates of aberrant salience in individuals at risk for psychosis [published correction appears in Schizphr Bull 2016; 42:1303].
      ). Similarly, using novel computational approaches to investigate the differences between patients with schizophrenia and control subjects in the implicit salience task, Katthagen et al. (
      • Katthagen T.
      • Mathys C.
      • Deserno L.
      • Walter H.
      • Kathmann N.
      • Heinz A.
      • Schlagenhauf F.
      Modeling subjective relevance in schizophrenia and its relation to aberrant salience.
      ) found that patients had a stronger bias toward irrelevant information, which was associated with stronger negative symptomatology and not with positive symptoms as conceptually expected. Partially, these inconsistencies might be explained by different medication or treatment statuses. Abboud et al. (
      • Abboud R.
      • Roiser J.P.
      • Khalifeh H.
      • Ali S.
      • Harrison I.
      • Killaspy H.T.
      • Joyce E.M.
      Are persistent delusions in schizophrenia associated with aberrant salience?.
      ), for example, showed that patients with treatment-resistant schizophrenia did not show heightened aberrant salience. This result may be explained by the nonelevated levels of dopamine synthesis capacity (
      • Demjaha A.
      • Murray R.M.
      • McGuire P.K.
      • Kapur S.
      • Howes O.D.
      Dopamine synthesis capacity in patients with treatment-resistant schizophrenia.
      ) in contrast to otherwise elevated levels in the early and chronic stages of the illness (
      • Howes O.D.
      • Kambeitz J.
      • Kim E.
      • Stahl D.
      • Slifstein M.
      • Abi-Dargham A.
      • Kapur S.
      The nature of dopamine dysfunction in schizophrenia and what this means for treatment.
      ). A further argument explaining the inconsistencies might be the investigation of task-irrelevant information in a rewarding setting, which might not allow a clear differentiation between the underlying processes.
      In a reward-independent learning setting, Ceaser and Barch (
      • Ceaser A.E.
      • Barch D.M.
      Striatal activity is associated with deficits of cognitive control and aberrant salience for patients with schizophrenia.
      ) found that during a cognitive control task, patients with schizophrenia were more likely to inappropriately encode irrelevant distracter stimuli, showing increased prefrontal and striatal activity. In emotion recognition studies, patients with chronic schizophrenia consistently showed increased brain signaling to neutral emotional stimuli, the irrelevant stimuli in the tasks, with effects being especially strong for face stimuli (
      • Potvin S.
      • Tikàsz A.
      • Mendrek A.
      Emotionally neutral stimuli are not neutral in schizophrenia: A mini review of functional neuroimaging studies.
      ). Similar results were also reported for emotional and neutral word processing; patients with schizophrenia and their unaffected first-degree relatives demonstrated increased attention to neutral words compared with control subjects (
      • Alfimova M.V.
      • Uvarova L.G.
      Changes in EEG spectral power on perception of neutral and emotional words in patients with schizophrenia, their relatives, and healthy subjects from the general population.
      ). The brain regions involved varied across studies and included the amygdala, prefrontal and cingulate subregions, and hippocampus (
      • Potvin S.
      • Tikàsz A.
      • Mendrek A.
      Emotionally neutral stimuli are not neutral in schizophrenia: A mini review of functional neuroimaging studies.
      ). Results in individuals at risk for psychosis were less consistent with regard to neutral stimuli (
      • Potvin S.
      • Tikàsz A.
      • Mendrek A.
      Emotionally neutral stimuli are not neutral in schizophrenia: A mini review of functional neuroimaging studies.
      ). These studies provide evidence for altered processing of neutral events in subcortical regions associated with dopaminergic dysregulation. When studying the processing of neutral stimuli in a reward-independent setting, there is more consistent evidence of a link between neural dysregulation and performance and symptom strength; increased striatal activity during incorrect distracter trials correlated positively with aberrant salience symptoms (
      • Ceaser A.E.
      • Barch D.M.
      Striatal activity is associated with deficits of cognitive control and aberrant salience for patients with schizophrenia.
      ). In a behavioral causal learning task, Morris et al. (
      • Morris R.
      • Griffiths O.
      • Le Pelley M.E.
      • Weickert T.W.
      Attention to irrelevant cues is related to positive symptoms in schizophrenia.
      ) showed that people with schizophrenia with severe positive symptoms failed to discriminate between predictive and nonpredictive cues compared with healthy adults. Furthermore, overweighting nonpredictive cues was correlated with more severe positive symptom scores in schizophrenia (
      • Morris R.
      • Griffiths O.
      • Le Pelley M.E.
      • Weickert T.W.
      Attention to irrelevant cues is related to positive symptoms in schizophrenia.
      ). A recent study (
      • Li L.Y.
      • Castro M.K.
      • Martin E.A.
      What you want may not be what you like: A test of the aberrant salience hypothesis in schizophrenia risk.
      ) exploring neutral stimuli in a reward learning setting showed that individuals with psychotic-like experiences overattribute salience to neutral stimuli and underattribute salience to rewards, indicating that abnormal salience attribution is a trait-like feature. Together, these studies show that neutral or irrelevant stimuli are consistently overweighted in patients at different disease stages, with the exception of patients with treatment-resistant schizophrenia, showing a clear indication of aberrant salience, although associations with symptoms are inconsistent.

      Aberrant Salience as Altered Processing of Relevant Information

      Aberrant brain processing of informative and relevant but nonmotivational salient events in psychosis has been reported in several studies. In a recent study, Knolle et al. (
      • Knolle F.
      • Ermakova A.O.
      • Justicia A.
      • Fletcher P.C.
      • Bunzeck N.
      • Düzel E.
      • Murray G.K.
      Brain responses to different types of salience in antipsychotic naïve first episode psychosis: An fMRI study.
      ) used a visual, passive oddball paradigm (
      • Bunzeck N.
      • Düzel E.
      Absolute coding of stimulus novelty in the human substantia nigra/VTA.
      ) to investigate novelty, negative emotional salience, and targetness, which required a button press, in patients with antipsychotic naïve first-episode psychosis. The patients exhibited reduced substantia nigra, ventral tegmental area, and striatal and cingulate signaling to novelty; reduced substantia nigra, ventral tegmental area, amygdala, and striatal and cingulate signaling to negative emotional salience; and reduced substantia nigra, ventral tegmental area, and cingulate signaling to targetness. Modinos et al. (
      • Modinos G.
      • Allen P.
      • Zugman A.
      • Dima D.
      • Azis M.
      • Samson C.
      • et al.
      Neural circuitry of novelty salience processing in psychosis risk: Association with clinical outcome.
      ), using the same paradigm, showed similar results for novelty processing in at-risk individuals using the same task. Similar results of altered salience processing have been reported in patients with Parkinson’s disease exhibiting psychotic, mainly hallucinatory, symptoms (
      • Knolle F.
      • Garofalo S.
      • Viviani R.
      • Justicia A.
      • Ermakova A.O.
      • Blank H.
      • et al.
      Altered subcortical emotional salience processing differentiates Parkinson’s patients with and without psychotic symptoms.
      ), again using the same paradigm (
      • Bunzeck N.
      • Düzel E.
      Absolute coding of stimulus novelty in the human substantia nigra/VTA.
      ). Moreover, patients with schizophrenia and early psychosis show deficits when processing emotions and intrinsic salient events, especially in the context of facial recognition (
      • Savla G.N.
      • Vella L.
      • Armstrong C.C.
      • Penn D.L.
      • Twamley E.W.
      Deficits in domains of social cognition in schizophrenia: A meta-analysis of the empirical evidence.
      ). In a positron emission tomography (PET) study, Taylor et al. (
      • Taylor S.F.
      • Phan K.L.
      • Britton J.C.
      • Liberzon I.
      Neural response to emotional salience in schizophrenia.
      ) showed impaired neuronal signaling in the ventral striatum in response to emotional salient events in people with chronic and acute psychosis. In general, experimental findings are less consistent. A study reporting overall increased arousal in patients with schizophrenia during processing of emotionally neutral and salient stimuli (
      • Haralanova E.
      • Haralanov S.
      • Beraldi A.
      • Möller H.J.
      • Hennig-Fast K.
      Subjective emotional over-arousal to neutral social scenes in paranoid schizophrenia.
      ) showed that the increase resulted solely from falsely attributing salience to neutral stimuli. This view has been confirmed by a meta-analysis showing similar processing of emotionally relevant information but attribution of aberrant salience to emotionally neutral information (
      • Llerena K.
      • Strauss G.P.
      • Cohen A.S.
      Looking at the other side of the coin: A meta-analysis of self-reported emotional arousal in people with schizophrenia.
      ).
      A recent electroencephalography study (
      • Tang Y.
      • Wang J.
      • Zhang T.
      • Xu L.
      • Qian Z.
      • Cui H.
      • et al.
      P300 as an index of transition to psychosis and of remission: Data from a clinical high risk for psychosis study and review of literature.
      ) using a P300 auditory oddball paradigm reported that reduced P300, which reflects impaired salience processing, indicated both transition to psychosis in at-risk individuals and transition to remission from psychotic symptoms. These studies show consistent findings for altered, mainly reduced, processing of salient stimuli compared with control subjects. Bringing both accounts together, a study by Boehme et al. (
      • Boehme R.
      • Deserno L.
      • Gleich T.
      • Katthagen T.
      • Pankow A.
      • Behr J.
      • et al.
      Aberrant salience is related to reduced reinforcement learning signals and elevated dopamine synthesis capacity in healthy adults.
      ) investigating healthy subjects showed that individual variability in aberrant salience measures related negatively to ventral striatal and prefrontal reward prediction error signals and, in an exploratory analysis, was found to be positively associated with nucleus accumbens presynaptic dopamine levels.

      Aberrant Salience and Symptomatology

      The literature provides evidence for aberrant processing of nonmotivational (and motivational) salience with regard to relevant and irrelevant events. Dysregulations in salience processing are associated with a range of brain areas in early psychosis and chronic schizophrenia (Figure 2). Here, we wish to argue that aberrant salience may explain positive and negative symptoms via different, although interrelated, mechanisms. As Maia and Frank (
      • Maia T.V.
      • Frank M.J.
      An integrative perspective on the role of dopamine in schizophrenia.
      ) discussed, the failure to distinguish between salient and nonsalient events may be reflected in the dysregulated activity of phasic firing of dopamine neurons. While increased phasic dopaminergic firing to a nonsalient event, such as a radio in the background, may lead to the attribution of attention to this otherwise irrelevant stimulus, this suddenly important information (content of the radio show) might be reinterpreted causing, e.g., delusional thinking. However, decreased dopaminergic phasic firing to a salient event, such as an unexpected positive emotional expression on a person’s face, may blunt the importance of this information. The inadequate interpretation or evaluation of the situation may be linked to, e.g., anhedonia. The aberrant processing of these informative or relevant (i.e., salient) events, as seen in blunted prediction errors to rewards (see Prediction Error Signaling in Psychosis) (
      • Murray G.K.
      • Corlett P.R.
      • Clark L.
      • Pessiglione M.
      • Blackwell A.D.
      • Honey G.
      • et al.
      Substantia nigra/ventral tegmental reward prediction error disruption in psychosis.
      ,
      • Ermakova A.O.
      • Knolle F.
      • Justicia A.
      • Bullmore E.T.
      • Jones P.B.
      • Robbins T.W.
      • et al.
      Abnormal reward prediction-error signalling in antipsychotic naive individuals with first-episode psychosis or clinical risk for psychosis.
      ) or decreased responses to relevant visual stimuli (
      • Knolle F.
      • Garofalo S.
      • Viviani R.
      • Justicia A.
      • Ermakova A.O.
      • Blank H.
      • et al.
      Altered subcortical emotional salience processing differentiates Parkinson’s patients with and without psychotic symptoms.
      ,
      • Dowd E.C.
      • Barch D.M.
      Pavlovian reward prediction and receipt in schizophrenia: Relationship to anhedonia.
      ), seem to relate to negative symptoms, such as anhedonia or lack of motivation (
      • Dowd E.C.
      • Barch D.M.
      Pavlovian reward prediction and receipt in schizophrenia: Relationship to anhedonia.
      ,
      • Strauss G.P.
      • Waltz J.A.
      • Gold J.M.
      A review of reward processing and motivational impairment in schizophrenia.
      ,
      • Waltz J.A.
      • Schweitzer J.B.
      • Gold J.M.
      • Kurup P.K.
      • Ross T.J.
      • Salmeron B.J.
      • et al.
      Patients with schizophrenia have a reduced neural response to both unpredictable and predictable primary reinforcers.
      ,
      • Katthagen T.
      • Kaminski J.
      • Heinz A.
      • Buchert R.
      • Schlagenhauf F.
      Striatal dopamine and reward prediction error signaling in unmedicated schizophrenia patients.
      ). In contrast, aberrant processing of irrelevant (i.e., nonsalient) events, such as neutral events in reward learning or oddball paradigms, seems to provide an explanation for positive symptoms (
      • Morris R.
      • Griffiths O.
      • Le Pelley M.E.
      • Weickert T.W.
      Attention to irrelevant cues is related to positive symptoms in schizophrenia.
      ,
      • Li L.Y.
      • Castro M.K.
      • Martin E.A.
      What you want may not be what you like: A test of the aberrant salience hypothesis in schizophrenia risk.
      ). We do note, however, some studies with contradictory symptom associations [e.g., (
      • Roiser J.P.
      • Stephan K.E.
      • Den Ouden H.E.M.
      • Barnes T.R.E.
      • Friston K.J.
      • Joyce E.M.
      Do patients with schizophrenia exhibit aberrant salience?.
      ,
      • Katthagen T.
      • Mathys C.
      • Deserno L.
      • Walter H.
      • Kathmann N.
      • Heinz A.
      • Schlagenhauf F.
      Modeling subjective relevance in schizophrenia and its relation to aberrant salience.
      ,
      • Smieskova R.
      • Roiser J.P.
      • Chaddock C.A.
      • Schmidt A.
      • Harrisberger F.
      • Bendfeldt K.
      • et al.
      Modulation of motivational salience processing during the early stages of psychosis.
      )].
      Figure thumbnail gr2
      Figure 2Salience areas and psychosis. Simplified diagram of key regions involved in salience processing and underlying problems observed in psychosis. See text for citations and details. dl, dorsolateral; vm, ventromedial; VTA, ventral tegmental area.

      Prediction Error Signaling in Psychosis

      Prediction error is the mismatch between expectation and outcome and is, as an intrinsically salient event, a key driver of learning (
      • Diederen K.M.J.
      • Fletcher P.C.
      Dopamine, prediction error and beyond.
      ). Several studies reported reduced midbrain, striatal, and/or cortical processing of reward prediction errors in psychosis, which may underpin aspects of the clinical manifestations of psychotic illness (
      • Murray G.K.
      • Corlett P.R.
      • Clark L.
      • Pessiglione M.
      • Blackwell A.D.
      • Honey G.
      • et al.
      Substantia nigra/ventral tegmental reward prediction error disruption in psychosis.
      ,
      • Ermakova A.O.
      • Knolle F.
      • Justicia A.
      • Bullmore E.T.
      • Jones P.B.
      • Robbins T.W.
      • et al.
      Abnormal reward prediction-error signalling in antipsychotic naive individuals with first-episode psychosis or clinical risk for psychosis.
      ,
      • Katthagen T.
      • Kaminski J.
      • Heinz A.
      • Buchert R.
      • Schlagenhauf F.
      Striatal dopamine and reward prediction error signaling in unmedicated schizophrenia patients.
      ,
      • Gradin V.B.
      • Kumar P.
      • Waiter G.
      • Ahearn T.
      • Stickle C.
      • Milders M.
      • et al.
      Expected value and prediction error abnormalities in depression and schizophrenia.
      ,
      • Morris R.W.
      • Vercammen A.
      • Lenroot R.
      • Moore L.
      • Langton J.M.
      • Short B.
      • et al.
      Disambiguating ventral striatum fMRI-related BOLD signal during reward prediction in schizophrenia.
      ,
      • Schlagenhauf F.
      • Huys Q.J.M.
      • Deserno L.
      • Rapp M.A.
      • Beck A.
      • Heinze H.J.
      • et al.
      Striatal dysfunction during reversal learning in unmedicated schizophrenia patients.
      ). Here, we will mainly focus on functional magnetic resonance imaging (fMRI) studies of reward prediction error; other closely related topics such as mismatch negativity, Kamin blocking, latent inhibition, and causal learning prediction error have been previously studied in schizophrenia and discussed elsewhere [e.g., (
      • Gray J.A.
      Integrating schizophrenia.
      ,
      • Corlett P.R.
      • Honey G.D.
      • Aitken M.R.F.
      • Dickinson A.
      • Shanks D.R.
      • Absalom A.R.
      • et al.
      Frontal responses during learning predict vulnerability to the psychotogenic effects of ketamine: Linking cognition, brain activity, and psychosis.
      )].

      fMRI Studies of Reward Prediction Error in Early Psychosis and At-Risk Patients

      In an early study, Murray et al. (
      • Murray G.K.
      • Corlett P.R.
      • Clark L.
      • Pessiglione M.
      • Blackwell A.D.
      • Honey G.
      • et al.
      Substantia nigra/ventral tegmental reward prediction error disruption in psychosis.
      ) demonstrated abnormalities in patient brain responses correlating with reward prediction error in the dopaminergic midbrain, in striatal and limbic regions, and in cortical regions such as the dorsolateral prefrontal cortex (PFC). Subsequent studies using different psychological paradigms have found similar abnormalities [e.g., (
      • Morris R.W.
      • Vercammen A.
      • Lenroot R.
      • Moore L.
      • Langton J.M.
      • Short B.
      • et al.
      Disambiguating ventral striatum fMRI-related BOLD signal during reward prediction in schizophrenia.
      ,
      • Schlagenhauf F.
      • Huys Q.J.M.
      • Deserno L.
      • Rapp M.A.
      • Beck A.
      • Heinze H.J.
      • et al.
      Striatal dysfunction during reversal learning in unmedicated schizophrenia patients.
      )], especially in the early stages of psychosis, including in at-risk states [e.g., (
      • Millman Z.B.
      • Gallagher K.
      • Demro C.
      • Schiffman J.
      • Reeves G.M.
      • Gold J.M.
      • et al.
      Evidence of reward system dysfunction in youth at clinical high-risk for psychosis from two event-related fMRI paradigms.
      )]. Ermakova et al. (
      • Ermakova A.O.
      • Knolle F.
      • Justicia A.
      • Bullmore E.T.
      • Jones P.B.
      • Robbins T.W.
      • et al.
      Abnormal reward prediction-error signalling in antipsychotic naive individuals with first-episode psychosis or clinical risk for psychosis.
      ) documented impaired subcortical (midbrain) reward prediction error signals in an antipsychotic-free early psychosis sample and showed that an at-risk group with mild psychotic symptoms had a degree of midbrain signaling abnormalities. Notably, there was dorsolateral PFC prediction error dysfunction in the early psychosis sample, with comparatively intact cortical function in the at-risk group with mild psychotic symptoms.

      Relevant Pharmacological and Molecular Imaging Results

      Although PET studies have demonstrated robustly that there is excessive dopaminergic striatal release in schizophrenia (
      • Howes O.D.
      • Bose S.K.
      • Turkheimer F.
      • Valli I.
      • Egerton A.
      • Valmaggia L.R.
      • et al.
      Dopamine synthesis capacity before onset of psychosis: A prospective [18F]-DOPA PET imaging study.
      ), neuroimaging studies have generally shown impaired striatal signaling in patients during learning, which may appear perplexing if striatal fMRI signals are considered a pure assay of dopamine release. This apparent paradox was addressed by Bernacer et al. (
      • Bernacer J.
      • Corlett P.R.
      • Ramachandra P.
      • McFarlane B.
      • Turner D.C.
      • Clark L.
      • et al.
      Methamphetamine-induced disruption of frontostriatal reward learning signals: Relation to psychotic symptoms.
      ), who showed that administration of methamphetamine (which floods the striatum with dopamine) to healthy volunteers leads to a disruption of striatal prediction error–associated activity. The degree to which methamphetamine induced mild psychotic experiences was related to the degree to which it disrupted the expected value signal in the ventromedial prefrontal and posterior cingulate cortices. The study showed that a drug that reliably increases dopamine release is not beneficial to neural or behavioral indices of learning, indicating that caution should be exercised when using fMRI signals as a clean readout of neurochemical processes. A related, nuanced finding is that antipsychotic dopamine D2 receptor antagonist medications may enhance brain activations during reward processing in patients, in contrast to their effects in studies in healthy individuals (
      • Nielsen M.O.
      • Rostrup E.
      • Wulff S.
      • Bak N.
      • Broberg B.V.
      • Lublin H.
      • et al.
      Improvement of brain reward abnormalities by antipsychotic monotherapy in schizophrenia.
      ).

      Precision of Prediction Error

      Recent interest has focused not simply on prediction error per se, but on the precision of prediction error, which theory posits should play an important role in belief updating under uncertainty. Simple models update value or beliefs in proportion to the prediction error, but it is thought that a prediction error of a given magnitude should influence belief updating depending on the degree of uncertainty with which it is estimated. Substantially updating beliefs because of an imprecisely estimated prediction error could be maladaptive (
      • Haarsma J.
      • Fletcher P.C.
      • Griffin J.D.
      • Taverne H.J.
      • Ziauddeen H.
      • Spencer T.J.
      • et al.
      Precision weighting of cortical unsigned prediction error signals benefits learning, is mediated by dopamine, and is impaired in psychosis [published correction appears in Mol Psychiatry 2021; 26:5334].
      ,
      • Diederen K.M.J.
      • Spencer T.
      • Vestergaard M.D.
      • Fletcher P.C.
      • Schultz W.
      Adaptive prediction error coding in the human midbrain and striatum facilitates behavioral adaptation and learning efficiency.
      ), and several authors have posited that the precision of the prediction error could be a key locus of dysfunction in psychosis (
      • Katthagen T.
      • Mathys C.
      • Deserno L.
      • Walter H.
      • Kathmann N.
      • Heinz A.
      • Schlagenhauf F.
      Modeling subjective relevance in schizophrenia and its relation to aberrant salience.
      ). Haarsma et al. (
      • Haarsma J.
      • Fletcher P.C.
      • Griffin J.D.
      • Taverne H.J.
      • Ziauddeen H.
      • Spencer T.J.
      • et al.
      Precision weighting of cortical unsigned prediction error signals benefits learning, is mediated by dopamine, and is impaired in psychosis [published correction appears in Mol Psychiatry 2021; 26:5334].
      ) found behavioral and brain imaging evidence that patients with early psychosis have learning abnormalities related to the degree of precision weighting of prediction error in the superior frontal cortex. They focused on unsigned prediction error (i.e., prediction error that indicates surprise without valence evaluation, being worse or better than expected), which is often associated with cortical brain activity in human fMRI studies (
      • Corlett P.R.
      • Mollick J.A.
      • Kober H.
      Substrates of human prediction error for incentives, perception, cognition, and action.
      ). The degree of cortical abnormality was most pronounced in early psychosis and linked to the severity of positive psychotic symptoms, with relatively less impaired cortical function in at-risk patients with fewer symptoms. This pattern fits with the relatively distinct roles of the cortical areas in modulating the level of certainty of an unsigned prediction error estimation, compared with the role of the subcortical areas in signaling signed prediction errors, and hints toward a key role for the cortical function in the progression from the at-risk state to the frank psychotic illness state (Figure 3).
      Figure thumbnail gr3
      Figure 3Prediction error areas and psychosis. Simplified diagram of key regions involved in prediction error signaling and underlying problems observed in psychosis. See text for citations and details. VTA, ventral tegmental area.

      Prediction Error Signals in Psychosis May Differ Across Illness Stages

      We do note that there have been inconsistencies in the literature of prediction error signaling in psychosis. This is reflected in the conflicting accounts of two meta-analyses, one of which found relatively little evidence of abnormal fMRI reward prediction error signals in patients with schizophrenia compared with control subjects, although there were differences between schizophrenia and depression (
      • Strauss G.P.
      • Waltz J.A.
      • Gold J.M.
      A review of reward processing and motivational impairment in schizophrenia.
      ,
      • Yaple Z.A.
      • Tolomeo S.
      • Yu R.
      Abnormal prediction error processing in schizophrenia and depression.
      ). Another meta-analysis did document striatal reward prediction error abnormalities in psychosis (
      • Radua J.
      • Schmidt A.
      • Borgwardt S.
      • Heinz A.
      • Schlagenhauf F.
      • McGuire P.
      • Fusar-Poli P.
      Ventral striatal activation during reward processing in psychosis: A neurofunctional meta-analysis.
      ). Some studies that do not show group differences in activation have shown relationships at the interindividual level between striatal activation and the severity of anhedonia (
      • Dowd E.C.
      • Barch D.M.
      Pavlovian reward prediction and receipt in schizophrenia: Relationship to anhedonia.
      ,
      • Waltz J.A.
      • Kasanova Z.
      • Ross T.J.
      • Salmeron B.J.
      • McMahon R.P.
      • Gold J.M.
      • Stein E.A.
      The roles of reward, default, and executive control networks in set-shifting impairments in schizophrenia.
      ). One theme that may be emerging is that predominantly medicated samples of patients with schizophrenia have relatively intact brain prediction error signals, especially in chronic illness [e.g., (
      • Hernaus D.
      • Xu Z.
      • Brown E.C.
      • Ruiz R.
      • Frank M.J.
      • Gold J.M.
      • Waltz J.A.
      Motivational deficits in schizophrenia relate to abnormalities in cortical learning rate signals.
      )], whereas wholly or partly unmedicated samples, especially of early psychosis or schizophrenia, often show brain reward prediction error abnormalities (
      • Knolle F.
      • Ermakova A.O.
      • Justicia A.
      • Fletcher P.C.
      • Bunzeck N.
      • Düzel E.
      • Murray G.K.
      Brain responses to different types of salience in antipsychotic naïve first episode psychosis: An fMRI study.
      ,
      • Murray G.K.
      • Corlett P.R.
      • Clark L.
      • Pessiglione M.
      • Blackwell A.D.
      • Honey G.
      • et al.
      Substantia nigra/ventral tegmental reward prediction error disruption in psychosis.
      ,
      • Katthagen T.
      • Kaminski J.
      • Heinz A.
      • Buchert R.
      • Schlagenhauf F.
      Striatal dopamine and reward prediction error signaling in unmedicated schizophrenia patients.
      ,
      • Schlagenhauf F.
      • Huys Q.J.M.
      • Deserno L.
      • Rapp M.A.
      • Beck A.
      • Heinze H.J.
      • et al.
      Striatal dysfunction during reversal learning in unmedicated schizophrenia patients.
      ), as has been seen previously in the related field of latent inhibition studies in psychosis (
      • Gray J.A.
      Integrating schizophrenia.
      ,
      • Martins Serra A.
      • Jones S.H.
      • Toone B.
      • Gray J.A.
      Impaired associative learning in chronic schizophrenics and their first-degree relatives: A study of latent inhibition and the Kamin blocking effect.
      ).

      Reward Processing in Psychosis

      Reward describes a range of processes relating to the calculation, computation, and attainment of positive outcomes (
      • Der-Avakian A.
      • Barnes S.A.
      • Markou A.
      • Pizzagalli D.A.
      Translational assessment of reward and motivational deficits in psychiatric disorders.
      ) (Table 2). Reward deficits feature in a variety of psychiatric disorders and are commonly associated with dopamine systems (
      • Wise R.A.
      Dopamine, learning and motivation.
      ,
      • Kesby J.P.
      • Chang A.
      • Markou A.
      • Semenova S.
      Modeling human methamphetamine use patterns in mice: Chronic and binge methamphetamine exposure, reward function and neurochemistry.
      ). Increased negative symptoms in people with psychosis have been associated with decreased function in the ventral striatum in particular (
      • Culbreth A.J.
      • Moran E.K.
      • Kandala S.
      • Westbrook A.
      • Barch D.M.
      Effort, avolition and motivational experience in schizophrenia: Analysis of behavioral and neuroimaging data with relationships to daily motivational experience.
      ), but with structural changes in the orbital frontal cortex (
      • Baaré W.F.
      • Hulshoff Pol H.E.
      • Hijman R.
      • Mali W.P.
      • Viergever M.A.
      • Kahn R.S.
      Volumetric analysis of frontal lobe regions in schizophrenia: Relation to cognitive function and symptomatology.
      ,
      • Lacerda A.L.T.
      • Hardan A.Y.
      • Yorbik O.
      • Vemulapalli M.
      • Prasad K.M.
      • Keshavan M.S.
      Morphology of the orbitofrontal cortex in first-episode schizophrenia: Relationship with negative symptomatology.
      ) and increased glutamate levels in the anterior cingulate in early psychosis (
      • Egerton A.
      • Brugger S.
      • Raffin M.
      • Barker G.J.
      • Lythgoe D.J.
      • McGuire P.K.
      • Stone J.M.
      Anterior cingulate glutamate levels related to clinical status following treatment in first-episode schizophrenia.
      ). However, a variety of brain areas are involved in reward, and understanding the discrete contributing processes is critical for an appropriate neurobiological interpretation. Growing evidence supports a specific set of problems in patients with psychosis, related primarily to motivation and reinforcement learning.
      Table 2Types of Reward Processes
      ProcessRole
      AnticipationIncreased attention and responsiveness to upcoming rewards. In general, reward anticipation is associated with activation of the striatum, amygdala, and thalamus, and if multiple choices are available, the orbitofrontal and ventromedial prefrontal cortices can also be recruited (
      • Oldham S.
      • Murawski C.
      • Fornito A.
      • Youssef G.
      • Yücel M.
      • Lorenzetti V.
      The anticipation and outcome phases of reward and loss processing: A neuroimaging meta-analysis of the monetary incentive delay task.
      ). Within the striatum, the nucleus accumbens is thought to code the expected value, with the dorsal striatum (caudate) more involved in selection, action, and choice (
      • Oldham S.
      • Murawski C.
      • Fornito A.
      • Youssef G.
      • Yücel M.
      • Lorenzetti V.
      The anticipation and outcome phases of reward and loss processing: A neuroimaging meta-analysis of the monetary incentive delay task.
      ). Preclinical and human studies indicate that dopamine function is important in reward anticipation (
      • Murray G.K.
      • Clark L.
      • Corlett P.R.
      • Blackwell A.D.
      • Cools R.
      • Jones P.B.
      • et al.
      Incentive motivation in first-episode psychosis: A behavioural study.
      ,
      • Barbano M.F.
      • Cador M.
      Opioids for hedonic experience and dopamine to get ready for it.
      ,
      • Juckel G.
      • Friedel E.
      • Koslowski M.
      • Witthaus H.
      • Özgürdal S.
      • Gudlowski Y.
      • et al.
      Ventral striatal activation during reward processing in subjects with ultra-high risk for schizophrenia.
      ), with increasing dopamine tending to increase anticipation toward future rewards (
      • Webber H.E.
      • Lopez-Gamundi P.
      • Stamatovich S.N.
      • de Wit H.
      • Wardle M.C.
      Using pharmacological manipulations to study the role of dopamine in human reward functioning: A review of studies in healthy adults.
      ).
      ValuationComparison of various reward outcomes. Reward valuation is important in multiple stages of decision-making processes. The areas involved in computing and comparing value include the orbitofrontal cortex, nucleus accumbens, amygdala, and ventromedial prefrontal cortex (
      • Griffiths K.R.
      • Morris R.W.
      • Balleine B.W.
      Translational studies of goal-directed action as a framework for classifying deficits across psychiatric disorders.
      ). Initially, a comparison of the various rewards that may be available is required before computing the effort-reward trade-off. Once an outcome is acquired, its value is again assessed and used to update our understanding of the relationships between actions, effort, and reward. This is critical for effective reinforcement learning and generating anticipatory or incentive motivation for future rewards.
      Effort and MotivationCalculation of the effort-reward trade-off. The dorsal striatum is potentially involved in the selection of low-effort choices (
      • Kurniawan I.T.
      • Seymour B.
      • Talmi D.
      • Yoshida W.
      • Chater N.
      • Dolan R.J.
      Choosing to make an effort: The role of striatum in signaling physical effort of a chosen action.
      ), whereas the nucleus accumbens and anterior cingulate cortex are critical in modulating effort-cost trade-offs (
      • Croxson P.L.
      • Walton M.E.
      • O’Reilly J.X.
      • Behrens T.E.J.
      • Rushworth M.F.S.
      Effort-based cost-benefit valuation and the human brain.
      ,
      • Cowen S.L.
      • Davis G.A.
      • Nitz D.A.
      Anterior cingulate neurons in the rat map anticipated effort and reward to their associated action sequences.
      ,
      • Salamone J.D.
      • Yohn S.E.
      • López-Cruz L.
      • San Miguel N.
      • Correa M.
      Activational and effort-related aspects of motivation: Neural mechanisms and implications for psychopathology.
      ). In healthy people, greater endogenous striatal dopamine function or dopamine-stimulating pharmacological manipulations increase the willingness to expend effort when pursuing rewards (
      • Michely J.
      • Viswanathan S.
      • Hauser T.U.
      • Delker L.
      • Dolan R.J.
      • Grefkes C.
      The role of dopamine in dynamic effort-reward integration.
      ,
      • Westbrook A.
      • van den Bosch R.
      • Määttä J.I.
      • Hofmans L.
      • Papadopetraki D.
      • Cools R.
      • Frank M.J.
      Dopamine promotes cognitive effort by biasing the benefits versus costs of cognitive work.
      ). Preclinical studies have demonstrated that dopamine function in the nucleus accumbens is important in generating the value of work (
      • Hamid A.A.
      • Pettibone J.R.
      • Mabrouk O.S.
      • Hetrick V.L.
      • Schmidt R.
      • Vander Weele C.M.
      • et al.
      Mesolimbic dopamine signals the value of work.
      ), although arguments that dopamine is primarily coding reward value with minimal coding for the required effort have also been put forth (
      • Gan J.O.
      • Walton M.E.
      • Phillips P.E.
      Dissociable cost and benefit encoding of future rewards by mesolimbic dopamine.
      ,
      • Walton M.E.
      • Bouret S.
      What is the relationship between dopamine and effort?.
      ).
      Outcome and Outcome-Specific DevaluationEncoding the presence or absence of a reward and the actual reward value. The areas involved in monitoring and encoding reward outcomes include the nucleus accumbens, orbitofrontal cortex, ventromedial prefrontal cortex, and amygdala (
      • Oldham S.
      • Murawski C.
      • Fornito A.
      • Youssef G.
      • Yücel M.
      • Lorenzetti V.
      The anticipation and outcome phases of reward and loss processing: A neuroimaging meta-analysis of the monetary incentive delay task.
      ), although this depends on the specific reward learning task parameters (
      • Koch K.
      • Schachtzabel C.
      • Wagner G.
      • Schikora J.
      • Schultz C.
      • Reichenbach J.R.
      • et al.
      Altered activation in association with reward-related trial-and-error learning in patients with schizophrenia.
      ). Outcome-specific devaluation is a test of goal-directed action, requiring a participant to adjust, or bias, their actions away from an outcome after it has been devalued. This requires effective reward valuation (one outcome is now less rewarding), reward comparison (between the two possible outcomes), and then using this information to guide action selection.
      Learning With a Focus on Reversal LearningIncorporating reward outcomes and experience to navigate future choices. Many approaches have been used to probe reward learning, but one of the most widely used in psychosis research is probabilistic reversal learning, which requires a participant to navigate trial-by-trial feedback to determine which of two stimuli is rewarded more often. The presence of misleading negative feedback on the better choice (often 80% reward probability), as well as positive feedback on the worse choice (often 20% reward probability), means that the participant cannot rely solely on a “follow the win” strategy. To further probe how adaptable reward learning is, after the participant has successfully demonstrated their knowledge of the correct choice (6–10 consecutive correct trials), the contingencies are reversed. The areas commonly involved in probabilistic reversal learning include the striatum, orbitofrontal cortex, and ventral prefrontal cortex (
      • Conn K.A.
      • Burne T.H.J.
      • Kesby J.P.
      Subcortical dopamine and cognition in schizophrenia: Looking beyond psychosis in preclinical models.
      ,
      • Kesby J.P.
      • Eyles D.W.
      • McGrath J.J.
      • Scott J.G.
      Dopamine, psychosis and schizophrenia: The widening gap between basic and clinical neuroscience.
      ). Preclinical studies have also shown that dopamine is important for probabilistic reversal learning. For example, systemic amphetamine (a dopamine stimulant) administration in rats alters punishment learning without affecting win-stay use (
      • Wong S.A.
      • Thapa R.
      • Badenhorst C.A.
      • Briggs A.R.
      • Sawada J.A.
      • Gruber A.J.
      Opposing effects of acute and chronic d-amphetamine on decision-making in rats.
      ). Yet, increasing phasic dopamine signaling in the nucleus accumbens during a choice can increase the tendency to win-stay, even if the choice itself was not rewarded (
      • Hamid A.A.
      • Pettibone J.R.
      • Mabrouk O.S.
      • Hetrick V.L.
      • Schmidt R.
      • Vander Weele C.M.
      • et al.
      Mesolimbic dopamine signals the value of work.
      ). Alternatively, inactivating the nucleus accumbens (shell subregion) decreases win-stay use (
      • Dalton G.L.
      • Phillips A.G.
      • Floresco S.B.
      Preferential involvement by nucleus accumbens shell in mediating probabilistic learning and reversal shifts.
      ). A range of cortical areas have also been implicated in reward and punishment learning. For example, inactivating or lesioning the prelimbic, infralimbic, and orbitofrontal cortex subregions have all been shown to alter reward or punishment learning during reversal learning in rodents and marmosets (
      • Clarke H.F.
      • Robbins T.W.
      • Roberts A.C.
      Lesions of the medial striatum in monkeys produce perseverative impairments during reversal learning similar to those produced by lesions of the orbitofrontal cortex.
      ,
      • Dalton G.L.
      • Wang N.Y.
      • Phillips A.G.
      • Floresco S.B.
      Multifaceted contributions by different regions of the orbitofrontal and medial prefrontal cortex to probabilistic reversal learning.
      ,
      • Verharen J.P.H.
      • den Ouden H.E.M.
      • Adan R.A.H.
      • Vanderschuren L.J.M.J.
      Modulation of value-based decision making behavior by subregions of the rat prefrontal cortex.
      ). However, punishment learning may be more sensitive to cortical modulation than reward learning (
      • Clarke H.F.
      • Robbins T.W.
      • Roberts A.C.
      Lesions of the medial striatum in monkeys produce perseverative impairments during reversal learning similar to those produced by lesions of the orbitofrontal cortex.
      ,
      • Verharen J.P.H.
      • den Ouden H.E.M.
      • Adan R.A.H.
      • Vanderschuren L.J.M.J.
      Modulation of value-based decision making behavior by subregions of the rat prefrontal cortex.
      ).
      Since its original classification, schizophrenia has been associated with anhedonia (
      • Kraepelin E.
      Dementia Praecox and Paraphrenia.
      ,
      • Bleuler E.
      Dementia Praecox or the Group of Schizophrenias.
      ), a deficit in the pleasure received from rewarding or emotional stimuli (
      • Kring A.M.
      • Germans M.K.
      Anhedonia.
      ). Anhedonia is a core feature of major depressive disorder (
      American Psychiatric Association
      Diagnostic and Statistical Manual of Mental Disorders, DSM-5.
      ), and in schizophrenia it has been thought to contribute to broad motivational deficits. For example, if a reward is perceived to be less valuable, then the effort-reward trade-off is biased toward inaction. However, a growing amount of evidence suggests that people with schizophrenia experience the same pleasure from positive emotional and hedonic outcomes (
      • Llerena K.
      • Strauss G.P.
      • Cohen A.S.
      Looking at the other side of the coin: A meta-analysis of self-reported emotional arousal in people with schizophrenia.
      ,
      • Gard D.E.
      • Kring A.M.
      • Gard M.G.
      • Horan W.P.
      • Green M.F.
      Anhedonia in schizophrenia: Distinctions between anticipatory and consummatory pleasure.
      ,
      • Vignapiano A.
      • Mucci A.
      • Ford J.
      • Montefusco V.
      • Plescia G.M.
      • Bucci P.
      • Galderisi S.
      Reward anticipation and trait anhedonia: An electrophysiological investigation in subjects with schizophrenia.
      ). Perceived levels of anhedonia may instead be impairments in other reward processes, such as the anticipatory motivation toward rewarding outcomes (
      • Gard D.E.
      • Kring A.M.
      • Gard M.G.
      • Horan W.P.
      • Green M.F.
      Anhedonia in schizophrenia: Distinctions between anticipatory and consummatory pleasure.
      ).

      Reward Anticipation

      Deficits in reward anticipation have consistently been observed before psychosis onset. For example, a meta-analysis (including six studies in those at risk) observed impairments in those at risk (
      • Radua J.
      • Schmidt A.
      • Borgwardt S.
      • Heinz A.
      • Schlagenhauf F.
      • McGuire P.
      • Fusar-Poli P.
      Ventral striatal activation during reward processing in psychosis: A neurofunctional meta-analysis.
      ), and others have observed similar deficits in early psychosis (
      • Murray G.K.
      • Clark L.
      • Corlett P.R.
      • Blackwell A.D.
      • Cools R.
      • Jones P.B.
      • et al.
      Incentive motivation in first-episode psychosis: A behavioural study.
      ). A combined fMRI and PET study in healthy individuals found that nucleus accumbens dopamine release during reward anticipation was associated with activation of the dorsal striatum, amygdala, hippocampus, and thalamus (
      • Schott B.H.
      • Minuzzi L.
      • Krebs R.M.
      • Elmenhorst D.
      • Lang M.
      • Winz O.H.
      • et al.
      Mesolimbic functional magnetic resonance imaging activations during reward anticipation correlate with reward-related ventral striatal dopamine release.
      ), suggesting that dopamine release in the nucleus accumbens may be causative in recruiting the necessary networks. Multiple meta-analyses have observed reduced activation of the striatum and anterior cingulate cortex in those with schizophrenia [e.g., (
      • Radua J.
      • Schmidt A.
      • Borgwardt S.
      • Heinz A.
      • Schlagenhauf F.
      • McGuire P.
      • Fusar-Poli P.
      Ventral striatal activation during reward processing in psychosis: A neurofunctional meta-analysis.
      ,
      • Leroy A.
      • Amad A.
      • D’Hondt F.
      • Pins D.
      • Jaafari N.
      • Thomas P.
      • Jardri R.
      Reward anticipation in schizophrenia: A coordinate-based meta-analysis.
      )]. Less striatal activation was associated with greater psychotic symptoms (even after controlling for antipsychotic dosage) (
      • Leroy A.
      • Amad A.
      • D’Hondt F.
      • Pins D.
      • Jaafari N.
      • Thomas P.
      • Jardri R.
      Reward anticipation in schizophrenia: A coordinate-based meta-analysis.
      ), whereas decreased activation of the nucleus accumbens in those at risk for and with chronic schizophrenia was associated with increased negative symptoms (and not with positive symptoms) (
      • Radua J.
      • Schmidt A.
      • Borgwardt S.
      • Heinz A.
      • Schlagenhauf F.
      • McGuire P.
      • Fusar-Poli P.
      Ventral striatal activation during reward processing in psychosis: A neurofunctional meta-analysis.
      ). There is some evidence that antipsychotic treatment can improve striatal signaling during reward anticipation in people with schizophrenia but only in those who show significantly decreased positive symptoms (
      • Wulff S.
      • Nielsen M.Ø.
      • Rostrup E.
      • Svarer C.
      • Jensen L.T.
      • Pinborg L.
      • Glenthøj B.Y.
      The relation between dopamine D2 receptor blockade and the brain reward system: A longitudinal study of first-episode schizophrenia patients.
      ) or when treated with atypical antipsychotics (
      • Juckel G.
      • Schlagenhauf F.
      • Koslowski M.
      • Filonov D.
      • Wüstenberg T.
      • Villringer A.
      • et al.
      Dysfunction of ventral striatal reward prediction in schizophrenic patients treated with typical, not atypical, neuroleptics.
      ). However, the relationship between antipsychotic treatment, negative symptoms, and reward anticipation may be more complicated (
      • Nielsen M.Ø.
      • Rostrup E.
      • Broberg B.V.
      • Wulff S.
      • Glenthøj B.
      Negative symptoms and reward disturbances in schizophrenia before and after antipsychotic monotherapy.
      ). Furthermore, impairments in reward anticipation may be an early developmental trait preceding psychosis onset. For example, Vink et al. (
      • Vink M.
      • de Leeuw M.
      • Pouwels R.
      • van den Munkhof H.E.
      • Kahn R.S.
      • Hillegers M.
      Diminishing striatal activation across adolescent development during reward anticipation in offspring of schizophrenia patients.
      ) found that nucleus accumbens activation during reward anticipation decreased across adolescence in the children of people with schizophrenia (i.e., carrying a higher familial risk of developing psychosis) but not in the children of healthy control subjects. Conversely, it has been suggested that reduced anticipatory motivation may be a byproduct of a decreased ability to accurately maintain value representation (
      • Gold J.M.
      • Waltz J.A.
      • Prentice K.J.
      • Morris S.E.
      • Heerey E.A.
      Reward processing in schizophrenia: A deficit in the representation of value.
      ) or reward learning, more generally (
      • Chase H.W.
      • Loriemi P.
      • Wensing T.
      • Eickhoff S.B.
      • Nickl-Jockschat T.
      Meta-analytic evidence for altered mesolimbic responses to reward in schizophrenia.
      ).

      Avolition and Effort-Reward Trade-offs

      Another primary negative symptom thought to reflect reward impairments is avolition, or a lack of willingness to do tasks required for achieving a goal. Decreased motivation is evident in self-reports in those with early psychosis (
      • Fervaha G.
      • Takeuchi H.
      • Foussias G.
      • Hahn M.K.
      • Agid O.
      • Remington G.
      Achievement motivation in early schizophrenia: Relationship with symptoms, cognition and functional outcome.
      ), and there is strong evidence that people with schizophrenia are less motivated and less willing to expend the same effort to attain rewards as healthy individuals (
      • Culbreth A.J.
      • Moran E.K.
      • Kandala S.
      • Westbrook A.
      • Barch D.M.
      Effort, avolition and motivational experience in schizophrenia: Analysis of behavioral and neuroimaging data with relationships to daily motivational experience.
      ,
      • Green M.F.
      • Horan W.P.
      • Barch D.M.
      • Gold J.M.
      Effort-based decision making: A novel approach for assessing motivation in schizophrenia.
      ). For example, people with schizophrenia reached breakpoint earlier on a progressive ratio task (
      • Wolf D.H.
      • Satterthwaite T.D.
      • Kantrowitz J.J.
      • Katchmar N.
      • Vandekar L.
      • Elliott M.A.
      • Ruparel K.
      Amotivation in schizophrenia: Integrated assessment with behavioral, clinical, and imaging measures.
      ). In the progressive ratio task, the effort required to get a reward increases with each reward delivery, and the breakpoint refers to the point at which participants decide that the effort-reward trade-off is no longer worthwhile. This performance deficit was associated with greater amotivation scores and decreased ventral striatal function (
      • Wolf D.H.
      • Satterthwaite T.D.
      • Kantrowitz J.J.
      • Katchmar N.
      • Vandekar L.
      • Elliott M.A.
      • Ruparel K.
      Amotivation in schizophrenia: Integrated assessment with behavioral, clinical, and imaging measures.
      ). Decreases in motivation and willingness to work toward goals may be specific to certain types of rewards and the required efforts. For example, those with early psychosis were less likely to select the high-effort, high-reward options than healthy control subjects (
      • Chang W.C.
      • Chu A.O.K.
      • Treadway M.T.
      • Strauss G.P.
      • Chan S.K.W.
      • Lee E.H.M.
      • et al.
      Effort-based decision-making impairment in patients with clinically stabilized first-episode psychosis and its relationship with amotivation and psychosocial functioning.
      ). Furthermore, in a task in which increasing the effort improved the chance of receiving a high or low reward, people with schizophrenia were willing to expend the same effort regardless of reward size, whereas healthy individuals heavily biased their effort to increase the chance of higher rewards (
      • Pretus C.
      • Bergé D.
      • Guell X.
      • Pérez V.
      • Vilarroya Ó.
      Brain activity and connectivity differences in reward value discrimination during effort computation in schizophrenia.
      ). This was associated with reduced functional changes in the caudate and anterior cingulate cortex during reward presentation and in the caudate during effort selection.

      Goal-Directed Actions and Reward

      In parallel to motivational and effort-based impairments, people with schizophrenia are less able to use reward information correctly when guiding their actions (
      • Gold J.M.
      • Waltz J.A.
      • Prentice K.J.
      • Morris S.E.
      • Heerey E.A.
      Reward processing in schizophrenia: A deficit in the representation of value.
      ). A good example is the result obtained when patients with schizophrenia carry out outcome-specific devaluation tasks (
      • Conn K.A.
      • Burne T.H.J.
      • Kesby J.P.
      Subcortical dopamine and cognition in schizophrenia: Looking beyond psychosis in preclinical models.
      ,
      • Suetani S.
      • Baker A.
      • Garner K.
      • Cosgrove P.
      • Mackay-Sim M.
      • Siskind D.
      • et al.
      Impairments in goal-directed action and reversal learning in a proportion of individuals with psychosis.
      ). Morris et al. (
      • Morris R.W.
      • Quail S.
      • Griffiths K.R.
      • Green M.J.
      • Balleine B.W.
      Corticostriatal control of goal-directed action is impaired in schizophrenia.
      ,
      • Morris R.W.
      • Cyrzon C.
      • Green M.J.
      • Le Pelley M.E.
      • Balleine B.W.
      Impairments in action–outcome learning in schizophrenia.
      ) demonstrated that people with schizophrenia were able to understand that one outcome was worth less after devaluation but failed to alter their actions in response to this information. Our work suggests that this may occur in a specific subgroup of those with chronic psychosis and is less likely to be observed in early psychosis (
      • Suetani S.
      • Baker A.
      • Garner K.
      • Cosgrove P.
      • Mackay-Sim M.
      • Siskind D.
      • et al.
      Impairments in goal-directed action and reversal learning in a proportion of individuals with psychosis.
      ). These goal-directed action impairments were due to the inability to correctly relate outcomes causally to actions rather than problems in reward valuation (
      • Morris R.W.
      • Cyrzon C.
      • Green M.J.
      • Le Pelley M.E.
      • Balleine B.W.
      Impairments in action–outcome learning in schizophrenia.
      ). Disruptions in caudate function, but not PFC function, were associated with the deficit in responding toward the more valuable outcome (
      • Morris R.W.
      • Quail S.
      • Griffiths K.R.
      • Green M.J.
      • Balleine B.W.
      Corticostriatal control of goal-directed action is impaired in schizophrenia.
      ). The observed decreases in caudate function during responses in those with schizophrenia were associated with increased negative symptom severity (including avolition) (
      • Morris R.W.
      • Quail S.
      • Griffiths K.R.
      • Green M.J.
      • Balleine B.W.
      Corticostriatal control of goal-directed action is impaired in schizophrenia.
      ). Furthermore, impairments in the ability to causally relate action-outcome associations were accompanied by increased overall disability scores (
      • Kesby J.P.
      • Eyles D.W.
      • McGrath J.J.
      • Scott J.G.
      Dopamine, psychosis and schizophrenia: The widening gap between basic and clinical neuroscience.
      ). Preclinical studies have highlighted that striatal dopamine is important in establishing causal action-outcome associations and in action selection (
      • Lex B.
      • Hauber W.
      The role of dopamine in the prelimbic cortex and the dorsomedial striatum in instrumental conditioning.
      ,
      • Howard C.D.
      • Li H.
      • Geddes C.E.
      • Jin X.
      Dynamic nigrostriatal dopamine biases action selection.
      ). Whether these impairments are evident at earlier disease stages is not known, but they likely reflect impairments in reward learning rather than reward valuation.

      Cognitive Control and Risk

      Imaging studies focused on cognitive control have indicated that across adolescence there is a marked improvement in our ability to increase cognitive performance directed at higher reward or risk. This may contribute to impairments in effort allocation in psychosis (
      • Culbreth A.J.
      • Moran E.K.
      • Barch D.M.
      Effort-cost decision-making in psychosis and depression: Could a similar behavioral deficit arise from disparate psychological and neural mechanisms?.
      ). Higher stakes recruit striatal, ventrolateral PFC, thalamic, and anterior cingulate areas more so than low-stakes trials (
      • Insel C.
      • Kastman E.K.
      • Glenn C.R.
      • Somerville L.H.
      Development of corticostriatal connectivity constrains goal-directed behavior during adolescence.
      ). Moreover, the nucleus accumbens functional coupling was greatest with the dorsal striatum in young adolescents but shifted to the ventrolateral PFC with increased age (
      • Insel C.
      • Kastman E.K.
      • Glenn C.R.
      • Somerville L.H.
      Development of corticostriatal connectivity constrains goal-directed behavior during adolescence.
      ). Model-based reward learning is positively associated with nucleus accumbens dopamine synthesis and activation of the nucleus accumbens and lateral PFC (
      • Deserno L.
      • Huys Q.J.M.
      • Boehme R.
      • Buchert R.
      • Heinze H.J.
      • Grace A.A.
      • et al.
      Ventral striatal dopamine reflects behavioral and neural signatures of model-based control during sequential decision making.
      ). In unmedicated patients with schizophrenia who follow similar reinforcement learning strategies as control subjects, decreased nucleus accumbens activation has been observed (
      • Schlagenhauf F.
      • Huys Q.J.M.
      • Deserno L.
      • Rapp M.A.
      • Beck A.
      • Heinze H.J.
      • et al.
      Striatal dysfunction during reversal learning in unmedicated schizophrenia patients.
      ), but in those who do not follow the same strategies (i.e., poorer performers), decreased activation of both the nucleus accumbens and ventrolateral PFC has been observed (
      • Schlagenhauf F.
      • Huys Q.J.M.
      • Deserno L.
      • Rapp M.A.
      • Beck A.
      • Heinze H.J.
      • et al.
      Striatal dysfunction during reversal learning in unmedicated schizophrenia patients.
      ), suggesting that model-based circuitry may be dysfunctional. In another study looking at high-reward/risk comparisons, caudate and dorsolateral prefrontal coupling increased with age (
      • Insel C.
      • Charifson M.
      • Somerville L.H.
      Neurodevelopmental shifts in learned value transfer on cognitive control during adolescence.
      ). This may indicate that when comparing reward values in choice situations, the maturation of corticostriatal systems critical for focusing cognitive effort are impaired or delayed in people with psychosis. Evidence of functional connectivity alterations in those with early psychosis and chronic schizophrenia demonstrates progressive deviation from healthy control subjects (
      • Li T.
      • Wang Q.
      • Zhang J.
      • Rolls E.T.
      • Yang W.
      • Palaniyappan L.
      • et al.
      Brain-wide analysis of functional connectivity in first-episode and chronic stages of schizophrenia.
      ), showing large alterations in functional connectivity in the thalamus, anterior cingulate cortex, and striatum (
      • Li T.
      • Wang Q.
      • Zhang J.
      • Rolls E.T.
      • Yang W.
      • Palaniyappan L.
      • et al.
      Brain-wide analysis of functional connectivity in first-episode and chronic stages of schizophrenia.
      ). However, other studies have demonstrated more widespread connectivity changes in unmedicated people with early psychosis, with changes in limbic circuits still pronounced (
      • Chopra S.
      • Francey S.M.
      • O’Donoghue B.
      • Sabaroedin K.
      • Arnatkeviciute A.
      • Cropley V.
      • et al.
      Functional connectivity in antipsychotic-treated and antipsychotic-naive patients with first-episode psychosis and low risk of self-harm or aggression: A secondary analysis of a randomized clinical trial.
      ). Improvements in thalamocortical connectivity were associated with antipsychotic treatment, suggesting that medication status is important when interpreting changes in functional connectivity.

      Reward Learning

      There is ample evidence that reward learning, or reinforcement learning, is altered in psychosis. Deficits in reward learning have been associated with higher general symptoms (
      • Betz L.T.
      • Brambilla P.
      • Ilankovic A.
      • Premkumar P.
      • Kim M.S.
      • Raffard S.
      • et al.
      Deciphering reward-based decision-making in schizophrenia: A meta-analysis and behavioral modeling of the Iowa Gambling Task.
      ) and negative symptoms (
      • Waltz J.A.
      • Kasanova Z.
      • Ross T.J.
      • Salmeron B.J.
      • McMahon R.P.
      • Gold J.M.
      • Stein E.A.
      The roles of reward, default, and executive control networks in set-shifting impairments in schizophrenia.
      ,
      • Reddy L.F.
      • Waltz J.A.
      • Green M.F.
      • Wynn J.K.
      • Horan W.P.
      Probabilistic reversal learning in schizophrenia: Stability of deficits and potential causal mechanisms.
      ). For example, people with schizophrenia tend to place more emphasis on immediate rewards, even when these choices are less advantageous over time (
      • Betz L.T.
      • Brambilla P.
      • Ilankovic A.
      • Premkumar P.
      • Kim M.S.
      • Raffard S.
      • et al.
      Deciphering reward-based decision-making in schizophrenia: A meta-analysis and behavioral modeling of the Iowa Gambling Task.
      ). Furthermore, psychosis is associated with jumping to conclusions, whereby those with psychosis are more likely to update their beliefs using less information [e.g., (
      • Henquet C.
      • van Os J.
      • Pries L.K.
      • Rauschenberg C.
      • Delespaul P.
      • Kenis G.
      • et al.
      A replication study of JTC bias, genetic liability for psychosis and delusional ideation.
      ,
      • Ermakova A.O.
      • Gileadi N.
      • Knolle F.
      • Justicia A.
      • Anderson R.
      • Fletcher P.C.
      • et al.
      Cost evaluation during decision-making in patients at early stages of psychosis.
      )]. A study in those at risk suggested that this reasoning bias develops with the onset of psychosis and may not be evident beforehand (
      • Catalan A.
      • Tognin S.
      • Kempton M.J.
      • Stahl D.
      • Salazar de Pablo G.
      • Nelson B.
      • et al.
      Relationship between jumping to conclusions and clinical outcomes in people at clinical high-risk for psychosis.
      ). Decreased activation of the nucleus accumbens, anterior cingulate cortex, and dorsolateral PFC has been associated with impaired reward learning in those with schizophrenia (
      • Chase H.W.
      • Loriemi P.
      • Wensing T.
      • Eickhoff S.B.
      • Nickl-Jockschat T.
      Meta-analytic evidence for altered mesolimbic responses to reward in schizophrenia.
      ,
      • Koch K.
      • Schachtzabel C.
      • Wagner G.
      • Schikora J.
      • Schultz C.
      • Reichenbach J.R.
      • et al.
      Altered activation in association with reward-related trial-and-error learning in patients with schizophrenia.
      ). One consistently reported behavioral tendency in schizophrenia is reduced win-stay use in reversal learning tasks (
      • Katthagen T.
      • Kaminski J.
      • Heinz A.
      • Buchert R.
      • Schlagenhauf F.
      Striatal dopamine and reward prediction error signaling in unmedicated schizophrenia patients.
      ,
      • Waltz J.A.
      • Kasanova Z.
      • Ross T.J.
      • Salmeron B.J.
      • McMahon R.P.
      • Gold J.M.
      • Stein E.A.
      The roles of reward, default, and executive control networks in set-shifting impairments in schizophrenia.
      ,
      • Suetani S.
      • Baker A.
      • Garner K.
      • Cosgrove P.
      • Mackay-Sim M.
      • Siskind D.
      • et al.
      Impairments in goal-directed action and reversal learning in a proportion of individuals with psychosis.
      ,
      • Reddy L.F.
      • Waltz J.A.
      • Green M.F.
      • Wynn J.K.
      • Horan W.P.
      Probabilistic reversal learning in schizophrenia: Stability of deficits and potential causal mechanisms.
      ,
      • Deserno L.
      • Boehme R.
      • Mathys C.
      • Katthagen T.
      • Kaminski J.
      • Stephan K.E.
      • et al.
      Volatility estimates increase choice switching and relate to prefrontal activity in schizophrenia.
      ). Win-stay refers to a participant selecting the same stimulus after winning a reward on the prior trial. Poorer reversal learning performance in unmedicated people with schizophrenia has been associated with decreased activation of the ventrolateral PFC and nucleus accumbens (
      • Schlagenhauf F.
      • Huys Q.J.M.
      • Deserno L.
      • Rapp M.A.
      • Beck A.
      • Heinze H.J.
      • et al.
      Striatal dysfunction during reversal learning in unmedicated schizophrenia patients.
      ). Decreased win-stay use has been observed in the Wisconsin Card Sorting Test (
      • Saperia S.
      • Da Silva S.
      • Siddiqui I.
      • Agid O.
      • Daskalakis Z.J.
      • Ravindran A.
      • et al.
      Reward-driven decision-making impairments in schizophrenia.
      ), which features a greater number of stimuli and contingencies. Deficits in social reward learning (
      • Hanssen E.
      • van Buuren M.
      • Van Atteveldt N.
      • Lemmers-Jansen I.L.
      • Fett A.J.
      Neural, behavioural and real-life correlates of social context sensitivity and social reward learning during interpersonal interactions in the schizophrenia spectrum.
      ) provide evidence of how these deficits can increase the functional burden of those with schizophrenia. Overall, it appears that people with schizophrenia are less able to use rewarding feedback to guide learning. Working memory deficits may also present as impaired reward learning processes, which has been demonstrated using task-based and computational modeling–based approaches in those with schizophrenia (
      • Pantelis C.
      • Wood S.J.
      • Proffitt T.M.
      • Testa R.
      • Mahony K.
      • Brewer W.J.
      • et al.
      Attentional set-shifting ability in first-episode and established schizophrenia: Relationship to working memory.
      ). However, other studies suggest that learning impairments in psychosis are often not explained by deficits in working memory (
      • Griffiths K.R.
      • Morris R.W.
      • Balleine B.W.
      Translational studies of goal-directed action as a framework for classifying deficits across psychiatric disorders.
      ,
      • Meyer-Lindenberg A.
      • Miletich R.S.
      • Kohn P.D.
      • Esposito G.
      • Carson R.E.
      • Quarantelli M.
      • et al.
      Reduced prefrontal activity predicts exaggerated striatal dopaminergic function in schizophrenia.
      ,
      • Kellendonk C.
      • Simpson E.H.
      • Polan H.J.
      • Malleret G.
      • Vronskaya S.
      • Winiger V.
      • et al.
      Transient and selective overexpression of dopamine D2 receptors in the striatum causes persistent abnormalities in prefrontal cortex functioning.
      ). These conflicting studies suggest that reinforcement learning deficits may not be a universal core trait in psychosis but rather a feature in a large subgroup of those with psychosis. Deficits in reinforcement learning are observed in those at risk for psychosis (
      • Millman Z.B.
      • Gallagher K.
      • Demro C.
      • Schiffman J.
      • Reeves G.M.
      • Gold J.M.
      • et al.
      Evidence of reward system dysfunction in youth at clinical high-risk for psychosis from two event-related fMRI paradigms.
      ) and with early psychosis (
      • Murray G.K.
      • Cheng F.
      • Clark L.
      • Barnett J.H.
      • Blackwell A.D.
      • Fletcher P.C.
      • et al.
      Reinforcement and reversal learning in first-episode psychosis.
      ,
      • Montagnese M.
      • Knolle F.
      • Haarsma J.
      • Griffin J.D.
      • Richards A.
      • Vertes P.E.
      • et al.
      Reinforcement learning as an intermediate phenotype in psychosis? Deficits sensitive to illness stage but not associated with polygenic risk of schizophrenia in the general population.
      ). Those at risk for psychosis exhibited less activation of the nucleus accumbens and ventromedial PFC in reward processing (
      • Millman Z.B.
      • Gallagher K.
      • Demro C.
      • Schiffman J.
      • Reeves G.M.
      • Gold J.M.
      • et al.
      Evidence of reward system dysfunction in youth at clinical high-risk for psychosis from two event-related fMRI paradigms.
      ). In contrast, reinforcement learning studies conducted in those with early psychosis suggest that deficits may include punishment learning, specifically a decreased sensitivity to punishment (
      • Suetani S.
      • Baker A.
      • Garner K.
      • Cosgrove P.
      • Mackay-Sim M.
      • Siskind D.
      • et al.
      Impairments in goal-directed action and reversal learning in a proportion of individuals with psychosis.
      ,
      • Montagnese M.
      • Knolle F.
      • Haarsma J.
      • Griffin J.D.
      • Richards A.
      • Vertes P.E.
      • et al.
      Reinforcement learning as an intermediate phenotype in psychosis? Deficits sensitive to illness stage but not associated with polygenic risk of schizophrenia in the general population.
      ). However, reversal learning impairments in early psychosis are less robust than in those with persistent schizophrenia, with some studies observing relatively intact performance (
      • Pantelis C.
      • Wood S.J.
      • Proffitt T.M.
      • Testa R.
      • Mahony K.
      • Brewer W.J.
      • et al.
      Attentional set-shifting ability in first-episode and established schizophrenia: Relationship to working memory.
      ,
      • Murray G.K.
      • Cheng F.
      • Clark L.
      • Barnett J.H.
      • Blackwell A.D.
      • Fletcher P.C.
      • et al.
      Reinforcement and reversal learning in first-episode psychosis.
      ). Clearly more work, including longitudinal studies, is required to determine what reversal learning indices in those at risk for psychosis or with early psychosis mean for subsequent outcomes (both diagnostically and in terms of treatment efficacy).

      Reward Systems in Psychosis

      Overall, schizophrenia is associated with a broad group of reward deficits spread across multiple brain areas (Figure 4). These include anticipatory and effort-related motivation, reward-based decision making, and reward learning. Nevertheless, all of these processes rely heavily on corticostriatal circuitry, which corroborates well-known alterations in striatal dopamine function (
      • Radua J.
      • Schmidt A.
      • Borgwardt S.
      • Heinz A.
      • Schlagenhauf F.
      • McGuire P.
      • Fusar-Poli P.
      Ventral striatal activation during reward processing in psychosis: A neurofunctional meta-analysis.
      ,
      • Chase H.W.
      • Loriemi P.
      • Wensing T.
      • Eickhoff S.B.
      • Nickl-Jockschat T.
      Meta-analytic evidence for altered mesolimbic responses to reward in schizophrenia.
      ,
      • Kesby J.P.
      • Eyles D.W.
      • McGrath J.J.
      • Scott J.G.
      Dopamine, psychosis and schizophrenia: The widening gap between basic and clinical neuroscience.
      ) and changes in cortical structure and function (
      • Conn K.A.
      • Burne T.H.J.
      • Kesby J.P.
      Subcortical dopamine and cognition in schizophrenia: Looking beyond psychosis in preclinical models.
      ,
      • Li T.
      • Wang Q.
      • Zhang J.
      • Rolls E.T.
      • Yang W.
      • Palaniyappan L.
      • et al.
      Brain-wide analysis of functional connectivity in first-episode and chronic stages of schizophrenia.
      ,
      • Chopra S.
      • Francey S.M.
      • O’Donoghue B.
      • Sabaroedin K.
      • Arnatkeviciute A.
      • Cropley V.
      • et al.
      Functional connectivity in antipsychotic-treated and antipsychotic-naive patients with first-episode psychosis and low risk of self-harm or aggression: A secondary analysis of a randomized clinical trial.
      ).
      Figure thumbnail gr4
      Figure 4Reward areas and psychosis. Simplified diagram of key regions involved in reward processing and underlying problems observed in psychosis. Impairments tend to be less obvious in earlier disease stages or, in the case of reward/punishment learning, may actually be in opposition. See text for citations and details.

      Salience and Reward Across the Psychosis Spectrum

      Our review is based heavily on cross-sectional data using different experimental paradigms. Although not surprising, this highlights the need for focused longitudinal studies to track how these processes change across illness stages. Nevertheless, a picture is emerging over several studies showing that cortical function during reward and salience processing is impaired in psychotic illness (established/chronic schizophrenia), especially in the ventral and dorsolateral PFC and the anterior cingulate cortex, but relatively spared in the earliest stages of psychosis (Figure 5). However, there is evidence of subcortical reward and salience dysfunction in at-risk patients, consistent with PET studies showing dopaminergic abnormalities, especially increased levels of striatal dopamine (
      • Howes O.D.
      • Bose S.K.
      • Turkheimer F.
      • Valli I.
      • Egerton A.
      • Valmaggia L.R.
      • et al.
      Dopamine synthesis capacity before onset of psychosis: A prospective [18F]-DOPA PET imaging study.
      ), early in the course of illness. In psychosis, antipsychotic medication may normalize some learning abnormalities and learning-related brain signals, such as prediction error signals (
      • Nielsen M.O.
      • Rostrup E.
      • Wulff S.
      • Bak N.
      • Broberg B.V.
      • Lublin H.
      • et al.
      Improvement of brain reward abnormalities by antipsychotic monotherapy in schizophrenia.
      ), potentially explaining the alleviating effects on positive symptoms. We suggest that subcortical reward and salience dysfunction may be an early manifestation of the illness, with cortical abnormalities in these domains becoming more prominent as the illness progresses. This contrasts with the hypotheses of schizophrenia that have proposed the primary abnormalities as being cortical in origin, which proceed and/or induce subcortical dopamine dysfunction (
      • Meyer-Lindenberg A.
      • Miletich R.S.
      • Kohn P.D.
      • Esposito G.
      • Carson R.E.
      • Quarantelli M.
      • et al.
      Reduced prefrontal activity predicts exaggerated striatal dopaminergic function in schizophrenia.
      ). However, our proposal is consistent with some animal models that show proof of principle for cortical dysfunction secondary to primary subcortical lesions (
      • Kellendonk C.
      • Simpson E.H.
      • Polan H.J.
      • Malleret G.
      • Vronskaya S.
      • Winiger V.
      • et al.
      Transient and selective overexpression of dopamine D2 receptors in the striatum causes persistent abnormalities in prefrontal cortex functioning.
      ) or developmental changes in subcortical dopamine systems (
      • Kesby J.P.
      • Cui X.
      • Burne T.H.J.
      • Eyles D.W.
      Altered dopamine ontogeny in the developmentally vitamin D deficient rat and its relevance to schizophrenia.
      ).
      Figure thumbnail gr5
      Figure 5Neurochemical and functional imaging associations with salience and reward processing in those at risk for and in early psychosis compared with (A) chronic psychosis and (B) schizophrenia. Colored arrows reflect key neurochemical changes, and black arrows indicate functional magnetic resonance imaging changes in those with psychosis and their associated behavior. Evidence of salience and reward impairments in those at risk for and in early psychosis are commonly found in subcortical structures, such as the associative striatum, nucleus accumbens, ventral tegmental area (VTA), and substantia nigra (SN). However, evidence of cortical glutamatergic abnormalities in the anterior cingulate cortex (ACC) have been observed. In contrast, there is evidence of widespread functional impairments in those with chronic psychosis or schizophrenia. This includes reduced functional connectivity in thalamocortical and corticostriatal projections (dashed lines) and reduced functional activity during salience and reward processing in the ACC, ventral prefrontal cortex (vPFC), and dorsolateral PFC (dlPFC). Therefore, it may be that subcortical alterations precede cortical impairments in driving negative symptoms and deficits in salience and reward processing.

      Translational Potential

      There are several potentially helpful new interventions at various stages of development, highly relevant for reward and salience processing domains in psychosis. For example, the behavioral intervention of cognitive remediation therapy has already been shown to be capable of modulating the aspects of reinforcement learning, such as sensitivity to rewards and punishments (
      • Cella M.
      • Bishara A.J.
      • Medin E.
      • Swan S.
      • Reeder C.
      • Wykes T.
      Identifying cognitive remediation change through computational modelling—Effects on reinforcement learning in schizophrenia.
      ). Moreover, reward learning can be used to improve attentiveness during conversational skill learning relevant for everyday functioning, improving outcomes (
      • Silverstein S.M.
      • Spaulding W.D.
      • Menditto A.A.
      • Savitz A.
      • Liberman R.P.
      • Berten S.
      • Starobin H.
      Attention shaping: A reward-based learning method to enhance skills training outcomes in schizophrenia.
      ). Pharmacological interventions at numerous molecular targets are of interest in the treatment of cognitive deficits in schizophrenia, including reinforcement learning domains, and could potentially be combined with cognitive remediation interventions (
      • Acheson D.T.
      • Twamley E.W.
      • Young J.W.
      Reward learning as a potential target for pharmacological augmentation of cognitive remediation for schizophrenia: A roadmap for preclinical development.
      ). A relevant line of inquiry in the rodent models advanced by Grace et al. (
      • Grace A.A.
      • Gomes F.V.
      The circuitry of dopamine system regulation and its disruption in schizophrenia: Insights into treatment and prevention.
      ) indicates that administration of prepubertal benzodiazepines mitigates the deleterious effect of perinatal or adolescent insults that otherwise lead to a hyperdopaminergic state directly relevant for salience processing. Recent advances in noninvasive brain stimulation also show potential for modulating brain circuits discussed in this article; for example, transcranial-focused ultrasound appears to be well tolerated in humans and has recently been shown to have the potential to target not only cortical but also subcortical structures and thus potentially to modulate brain activations in networks throughout the brain (
      • Cain J.A.
      • Visagan S.
      • Johnson M.A.
      • Crone J.
      • Blades R.
      • Spivak N.M.
      • et al.
      Real time and delayed effects of subcortical low intensity focused ultrasound.
      ). fMRI-based techniques that draw on real-time fMRI signal decoding and neurofeedback are under investigation regarding their ability to influence various psychological states, including addressing symptom domains in schizophrenia [e.g., reduction of auditory hallucinations (
      • Humpston C.
      • Garrison J.
      • Orlov N.
      • Aleman A.
      • Jardri R.
      • Fernyhough C.
      • Allen P.
      Real-time functional magnetic resonance imaging neurofeedback for the relief of distressing auditory-verbal hallucinations: Methodological and empirical advances.
      )], and merit further investigation. To fully realize the potential benefits of novel brain stimulation technology, we need to accelerate our understanding of the causal relationships between brain circuits and behavior in patients.

      Conclusions

      Together, salience and reward processes are integral to engaging our attention, stimulating anticipation of future events, and driving goal-directed behaviors. In this review, we have summarized the current knowledge of behavioral and functional neuroimaging in salience, reward, and prediction error. Although they are specific processes, they interact in multiple feedback and feedforward systems essential for decision making and cognition more generally. Further studies focused on subcortical systems during adolescence and the transition to psychosis are warranted.

      Acknowledgments and Disclosures

      This work was supported by the National Health and Medical Research Council (Grant No. APP1139960 [to JPK]), the Brain & Behavior Research Foundation (2019 Maltz Prize [to JPK]), and the European Union’s Horizon 2020 (Grant No. 754462 [to FK]).
      The authors report no biomedical financial interests or potential conflicts of interest.

      References

        • McCutcheon R.A.
        • Reis Marques T.
        • Howes O.D.
        Schizophrenia-An overview.
        JAMA Psychiatry. 2020; 77: 201-210
        • Howes O.D.
        • Egerton A.
        • Allan V.
        • McGuire P.
        • Stokes P.
        • Kapur S.
        Mechanisms underlying psychosis and antipsychotic treatment response in schizophrenia: Insights from PET and SPECT imaging.
        Curr Pharm Des. 2009; 15: 2550-2559
        • Howes O.D.
        • Montgomery A.J.
        • Asselin M.C.
        • Murray R.M.
        • Valli I.
        • Tabraham P.
        • et al.
        Elevated striatal dopamine function linked to prodromal signs of schizophrenia.
        Arch Gen Psychiatry. 2009; 66: 13-20
        • Kapur S.
        • Mamo D.
        Half a century of antipsychotics and still a central role for dopamine D2 receptors.
        Prog Neuropsychopharmacol Biol Psychiatry. 2003; 27: 1081-1090
        • Nucifora Jr., F.C.
        • Woznica E.
        • Lee B.J.
        • Cascella N.
        • Sawa A.
        Treatment resistant schizophrenia: Clinical, biological, and therapeutic perspectives.
        Neurobiol Dis. 2019; 131: 104257
        • Demjaha A.
        • Egerton A.
        • Murray R.M.
        • Kapur S.
        • Howes O.D.
        • Stone J.M.
        • McGuire P.K.
        Antipsychotic treatment resistance in schizophrenia associated with elevated glutamate levels but normal dopamine function.
        Biol Psychiatry. 2014; 75: e11-e13
        • Uno Y.
        • Coyle J.T.
        Glutamate hypothesis in schizophrenia.
        Psychiatry Clin Neurosci. 2019; 73: 204-215
        • Winton-Brown T.T.
        • Fusar-Poli P.
        • Ungless M.A.
        • Howes O.D.
        Dopaminergic basis of salience dysregulation in psychosis.
        Trends Neurosci. 2014; 37: 85-94
        • Abboud R.
        • Roiser J.P.
        • Khalifeh H.
        • Ali S.
        • Harrison I.
        • Killaspy H.T.
        • Joyce E.M.
        Are persistent delusions in schizophrenia associated with aberrant salience?.
        Schizophr Res Cogn. 2016; 4: 32-38
        • Murray G.K.
        • Corlett P.R.
        • Fletcher P.C.
        The neural underpinnings of associative learning in health and psychosis: How can performance be preserved when brain responses are abnormal?.
        Schizophr Bull. 2010; 36: 465-471
        • Corlett P.R.
        • Murray G.K.
        • Honey G.D.
        • Aitken M.R.F.
        • Shanks D.R.
        • Robbins T.W.
        • et al.
        Disrupted prediction-error signal in psychosis: Evidence for an associative account of delusions.
        Brain. 2007; 130: 2387-2400
        • Maia T.V.
        • Frank M.J.
        An integrative perspective on the role of dopamine in schizophrenia.
        Biol Psychiatry. 2017; 81: 52-66
        • Howes O.D.
        • Nour M.M.
        Dopamine and the aberrant salience hypothesis of schizophrenia.
        World Psychiatry. 2016; 15: 3-4
        • Kapur S.
        Psychosis as a state of aberrant salience: A framework linking biology, phenomenology, and pharmacology in schizophrenia.
        Am J Psychiatry. 2003; 160: 13-23
        • Heinz A.
        Dopaminergic dysfunction in alcoholism and schizophrenia—Psychopathological and behavioral correlates.
        Eur Psychiatry. 2002; 17: 9-16
        • Fusar-Poli P.
        • Meyer-Lindenberg A.
        Striatal presynaptic dopamine in schizophrenia, part II: Meta-analysis of [(18)F/(11)C]-DOPA PET studies.
        Schizophr Bull. 2013; 39: 33-42
        • Corlett P.R.
        • Honey G.D.
        • Fletcher P.C.
        From prediction error to psychosis: Ketamine as a pharmacological model of delusions.
        J Psychopharmacol. 2007; 21: 238-252
        • Schultz W.
        Predictive reward signal of dopamine neurons.
        J Neurophysiol. 1998; 80: 1-27
        • Schultz W.
        Dopamine reward prediction-error signalling: A two-component response.
        Nat Rev Neurosci. 2016; 17: 183-195
        • Horvitz J.C.
        Mesolimbocortical and nigrostriatal dopamine responses to salient non− reward events.
        Neuroscience. 2000; 96: 651-656
        • Bunzeck N.
        • Doeller C.F.
        • Dolan R.J.
        • Duzel E.
        Contextual interaction between novelty and reward processing within the mesolimbic system.
        Hum Brain Mapp. 2012; 33: 1309-1324
        • Haarsma J.
        • Fletcher P.C.
        • Griffin J.D.
        • Taverne H.J.
        • Ziauddeen H.
        • Spencer T.J.
        • et al.
        Precision weighting of cortical unsigned prediction error signals benefits learning, is mediated by dopamine, and is impaired in psychosis [published correction appears in Mol Psychiatry 2021; 26:5334].
        Mol Psychiatry. 2021; 26: 5320-5333
        • Valdés-Baizabal C.
        • Carbajal G.V.
        • Pérez-González D.
        • Malmierca M.S.
        Dopamine modulates subcortical responses to surprising sounds [published correction appears in PLoS Biol 2020; 18:e3000984].
        PLoS Biol. 2020; 18e3000744
        • Bromberg-Martin E.S.
        • Matsumoto M.
        • Hikosaka O.
        Dopamine in motivational control: Rewarding, aversive, and alerting.
        Neuron. 2010; 68: 815-834
        • Roiser J.P.
        • Stephan K.E.
        • Den Ouden H.E.M.
        • Barnes T.R.E.
        • Friston K.J.
        • Joyce E.M.
        Do patients with schizophrenia exhibit aberrant salience?.
        Psychol Med. 2009; 39: 199-209
        • Roiser J.P.
        • Howes O.D.
        • Chaddock C.A.
        • Joyce E.M.
        • McGuire P.
        Neural and behavioral correlates of aberrant salience in individuals at risk for psychosis [published correction appears in Schizphr Bull 2016; 42:1303].
        Schizophr Bull. 2013; 39: 1328-1336
        • Katthagen T.
        • Mathys C.
        • Deserno L.
        • Walter H.
        • Kathmann N.
        • Heinz A.
        • Schlagenhauf F.
        Modeling subjective relevance in schizophrenia and its relation to aberrant salience.
        PLoS Comput Biol. 2018; 14e1006319
        • Demjaha A.
        • Murray R.M.
        • McGuire P.K.
        • Kapur S.
        • Howes O.D.
        Dopamine synthesis capacity in patients with treatment-resistant schizophrenia.
        Am J Psychiatry. 2012; 169: 1203-1210
        • Howes O.D.
        • Kambeitz J.
        • Kim E.
        • Stahl D.
        • Slifstein M.
        • Abi-Dargham A.
        • Kapur S.
        The nature of dopamine dysfunction in schizophrenia and what this means for treatment.
        Arch Gen Psychiatry. 2012; 69: 776-786
        • Ceaser A.E.
        • Barch D.M.
        Striatal activity is associated with deficits of cognitive control and aberrant salience for patients with schizophrenia.
        Front Hum Neurosci. 2016; 9: 687
        • Potvin S.
        • Tikàsz A.
        • Mendrek A.
        Emotionally neutral stimuli are not neutral in schizophrenia: A mini review of functional neuroimaging studies.
        Front Psychiatry. 2016; 7: 115
        • Alfimova M.V.
        • Uvarova L.G.
        Changes in EEG spectral power on perception of neutral and emotional words in patients with schizophrenia, their relatives, and healthy subjects from the general population.
        Neurosci Behav Physiol. 2008; 38: 533-540
        • Morris R.
        • Griffiths O.
        • Le Pelley M.E.
        • Weickert T.W.
        Attention to irrelevant cues is related to positive symptoms in schizophrenia.
        Schizophr Bull. 2013; 39: 575-582
        • Li L.Y.
        • Castro M.K.
        • Martin E.A.
        What you want may not be what you like: A test of the aberrant salience hypothesis in schizophrenia risk.
        Cogn Affect Behav Neurosci. 2020; 20: 873-887
        • Knolle F.
        • Ermakova A.O.
        • Justicia A.
        • Fletcher P.C.
        • Bunzeck N.
        • Düzel E.
        • Murray G.K.
        Brain responses to different types of salience in antipsychotic naïve first episode psychosis: An fMRI study.
        Transl Psychiatry. 2018; 8: 196
        • Bunzeck N.
        • Düzel E.
        Absolute coding of stimulus novelty in the human substantia nigra/VTA.
        Neuron. 2006; 51: 369-379
        • Modinos G.
        • Allen P.
        • Zugman A.
        • Dima D.
        • Azis M.
        • Samson C.
        • et al.
        Neural circuitry of novelty salience processing in psychosis risk: Association with clinical outcome.
        Schizophr Bull. 2020; 46: 670-679
        • Knolle F.
        • Garofalo S.
        • Viviani R.
        • Justicia A.
        • Ermakova A.O.
        • Blank H.
        • et al.
        Altered subcortical emotional salience processing differentiates Parkinson’s patients with and without psychotic symptoms.
        Neuroimage Clin. 2020; 27: 102277
        • Savla G.N.
        • Vella L.
        • Armstrong C.C.
        • Penn D.L.
        • Twamley E.W.
        Deficits in domains of social cognition in schizophrenia: A meta-analysis of the empirical evidence.
        Schizophr Bull. 2013; 39: 979-992
        • Taylor S.F.
        • Phan K.L.
        • Britton J.C.
        • Liberzon I.
        Neural response to emotional salience in schizophrenia.
        Neuropsychopharmacology. 2005; 30: 984-995
        • Haralanova E.
        • Haralanov S.
        • Beraldi A.
        • Möller H.J.
        • Hennig-Fast K.
        Subjective emotional over-arousal to neutral social scenes in paranoid schizophrenia.
        Eur Arch Psychiatry Clin Neurosci. 2012; 262: 59-68
        • Llerena K.
        • Strauss G.P.
        • Cohen A.S.
        Looking at the other side of the coin: A meta-analysis of self-reported emotional arousal in people with schizophrenia.
        Schizophr Res. 2012; 142: 65-70
        • Tang Y.
        • Wang J.
        • Zhang T.
        • Xu L.
        • Qian Z.
        • Cui H.
        • et al.
        P300 as an index of transition to psychosis and of remission: Data from a clinical high risk for psychosis study and review of literature.
        Schizophr Res. 2020; 226: 74-83
        • Boehme R.
        • Deserno L.
        • Gleich T.
        • Katthagen T.
        • Pankow A.
        • Behr J.
        • et al.
        Aberrant salience is related to reduced reinforcement learning signals and elevated dopamine synthesis capacity in healthy adults.
        J Neurosci. 2015; 35: 10103-10111
        • Murray G.K.
        • Corlett P.R.
        • Clark L.
        • Pessiglione M.
        • Blackwell A.D.
        • Honey G.
        • et al.
        Substantia nigra/ventral tegmental reward prediction error disruption in psychosis.
        Mol Psychiatry. 2008; 13 (267–276): 239
        • Ermakova A.O.
        • Knolle F.
        • Justicia A.
        • Bullmore E.T.
        • Jones P.B.
        • Robbins T.W.
        • et al.
        Abnormal reward prediction-error signalling in antipsychotic naive individuals with first-episode psychosis or clinical risk for psychosis.
        Neuropsychopharmacology. 2018; 43: 1691-1699
        • Dowd E.C.
        • Barch D.M.
        Pavlovian reward prediction and receipt in schizophrenia: Relationship to anhedonia.
        PLoS One. 2012; 7e35622
        • Strauss G.P.
        • Waltz J.A.
        • Gold J.M.
        A review of reward processing and motivational impairment in schizophrenia.
        Schizophr Bull. 2014; 40: S107-S116
        • Waltz J.A.
        • Schweitzer J.B.
        • Gold J.M.
        • Kurup P.K.
        • Ross T.J.
        • Salmeron B.J.
        • et al.
        Patients with schizophrenia have a reduced neural response to both unpredictable and predictable primary reinforcers.
        Neuropsychopharmacology. 2009; 34: 1567-1577
        • Katthagen T.
        • Kaminski J.
        • Heinz A.
        • Buchert R.
        • Schlagenhauf F.
        Striatal dopamine and reward prediction error signaling in unmedicated schizophrenia patients.
        Schizophr Bull. 2020; 46: 1535-1546
        • Smieskova R.
        • Roiser J.P.
        • Chaddock C.A.
        • Schmidt A.
        • Harrisberger F.
        • Bendfeldt K.
        • et al.
        Modulation of motivational salience processing during the early stages of psychosis.
        Schizophr Res. 2015; 166: 17-23
        • Diederen K.M.J.
        • Fletcher P.C.
        Dopamine, prediction error and beyond.
        Neuroscientist. 2021; 27: 30-46
        • Gradin V.B.
        • Kumar P.
        • Waiter G.
        • Ahearn T.
        • Stickle C.
        • Milders M.
        • et al.
        Expected value and prediction error abnormalities in depression and schizophrenia.
        Brain. 2011; 134: 1751-1764
        • Morris R.W.
        • Vercammen A.
        • Lenroot R.
        • Moore L.
        • Langton J.M.
        • Short B.
        • et al.
        Disambiguating ventral striatum fMRI-related BOLD signal during reward prediction in schizophrenia.
        Mol Psychiatry. 2012; 17 (280–289): 235
        • Schlagenhauf F.
        • Huys Q.J.M.
        • Deserno L.
        • Rapp M.A.
        • Beck A.
        • Heinze H.J.
        • et al.
        Striatal dysfunction during reversal learning in unmedicated schizophrenia patients.
        Neuroimage. 2014; 89: 171-180
        • Gray J.A.
        Integrating schizophrenia.
        Schizophr Bull. 1998; 24: 249-266
        • Corlett P.R.
        • Honey G.D.
        • Aitken M.R.F.
        • Dickinson A.
        • Shanks D.R.
        • Absalom A.R.
        • et al.
        Frontal responses during learning predict vulnerability to the psychotogenic effects of ketamine: Linking cognition, brain activity, and psychosis.
        Arch Gen Psychiatry. 2006; 63: 611-621
        • Millman Z.B.
        • Gallagher K.
        • Demro C.
        • Schiffman J.
        • Reeves G.M.
        • Gold J.M.
        • et al.
        Evidence of reward system dysfunction in youth at clinical high-risk for psychosis from two event-related fMRI paradigms.
        Schizophr Res. 2020; 226: 111-119
        • Howes O.D.
        • Bose S.K.
        • Turkheimer F.
        • Valli I.
        • Egerton A.
        • Valmaggia L.R.
        • et al.
        Dopamine synthesis capacity before onset of psychosis: A prospective [18F]-DOPA PET imaging study.
        Am J Psychiatry. 2011; 168: 1311-1317
        • Bernacer J.
        • Corlett P.R.
        • Ramachandra P.
        • McFarlane B.
        • Turner D.C.
        • Clark L.
        • et al.
        Methamphetamine-induced disruption of frontostriatal reward learning signals: Relation to psychotic symptoms.
        Am J Psychiatry. 2013; 170: 1326-1334
        • Nielsen M.O.
        • Rostrup E.
        • Wulff S.
        • Bak N.
        • Broberg B.V.
        • Lublin H.
        • et al.
        Improvement of brain reward abnormalities by antipsychotic monotherapy in schizophrenia.
        Arch Gen Psychiatry. 2012; 69: 1195-1204
        • Diederen K.M.J.
        • Spencer T.
        • Vestergaard M.D.
        • Fletcher P.C.
        • Schultz W.
        Adaptive prediction error coding in the human midbrain and striatum facilitates behavioral adaptation and learning efficiency.
        Neuron. 2016; 90: 1127-1138
        • Corlett P.R.
        • Mollick J.A.
        • Kober H.
        Substrates of human prediction error for incentives, perception, cognition, and action.
        PsyArXiv. 2021; https://doi.org/10.31234/osf.io/pf89k
        • Yaple Z.A.
        • Tolomeo S.
        • Yu R.
        Abnormal prediction error processing in schizophrenia and depression.
        Hum Brain Mapp. 2021; 42: 3547-3560
        • Radua J.
        • Schmidt A.
        • Borgwardt S.
        • Heinz A.
        • Schlagenhauf F.
        • McGuire P.
        • Fusar-Poli P.
        Ventral striatal activation during reward processing in psychosis: A neurofunctional meta-analysis.
        JAMA Psychiatry. 2015; 72: 1243-1251
        • Waltz J.A.
        • Kasanova Z.
        • Ross T.J.
        • Salmeron B.J.
        • McMahon R.P.
        • Gold J.M.
        • Stein E.A.
        The roles of reward, default, and executive control networks in set-shifting impairments in schizophrenia.
        PLoS One. 2013; 8e57257
        • Hernaus D.
        • Xu Z.
        • Brown E.C.
        • Ruiz R.
        • Frank M.J.
        • Gold J.M.
        • Waltz J.A.
        Motivational deficits in schizophrenia relate to abnormalities in cortical learning rate signals.
        Cogn Affect Behav Neurosci. 2018; 18: 1338-1351
        • Martins Serra A.
        • Jones S.H.
        • Toone B.
        • Gray J.A.
        Impaired associative learning in chronic schizophrenics and their first-degree relatives: A study of latent inhibition and the Kamin blocking effect.
        Schizophr Res. 2001; 48: 273-289
        • Der-Avakian A.
        • Barnes S.A.
        • Markou A.
        • Pizzagalli D.A.
        Translational assessment of reward and motivational deficits in psychiatric disorders.
        in: Robbins T.W. Sahakian B.J. Translational Neuropsychopharmacology. Springer International Publishing, Cham, Germany2016: 231-262
        • Wise R.A.
        Dopamine, learning and motivation.
        Nat Rev Neurosci. 2004; 5: 483-494
        • Kesby J.P.
        • Chang A.
        • Markou A.
        • Semenova S.
        Modeling human methamphetamine use patterns in mice: Chronic and binge methamphetamine exposure, reward function and neurochemistry.
        Addict Biol. 2018; 23: 206-218
        • Culbreth A.J.
        • Moran E.K.
        • Kandala S.
        • Westbrook A.
        • Barch D.M.
        Effort, avolition and motivational experience in schizophrenia: Analysis of behavioral and neuroimaging data with relationships to daily motivational experience.
        Clin Psychol Sci. 2020; 8: 555-568
        • Baaré W.F.
        • Hulshoff Pol H.E.
        • Hijman R.
        • Mali W.P.
        • Viergever M.A.
        • Kahn R.S.
        Volumetric analysis of frontal lobe regions in schizophrenia: Relation to cognitive function and symptomatology.
        Biol Psychiatry. 1999; 45: 1597-1605
        • Lacerda A.L.T.
        • Hardan A.Y.
        • Yorbik O.
        • Vemulapalli M.
        • Prasad K.M.
        • Keshavan M.S.
        Morphology of the orbitofrontal cortex in first-episode schizophrenia: Relationship with negative symptomatology.
        Prog Neuropsychopharmacol Biol Psychiatry. 2007; 31: 510-516
        • Egerton A.
        • Brugger S.
        • Raffin M.
        • Barker G.J.
        • Lythgoe D.J.
        • McGuire P.K.
        • Stone J.M.
        Anterior cingulate glutamate levels related to clinical status following treatment in first-episode schizophrenia.
        Neuropsychopharmacology. 2012; 37: 2515-2521
        • Kraepelin E.
        Dementia Praecox and Paraphrenia.
        Chicago Medical Book Company, Chicago1919
        • Bleuler E.
        Dementia Praecox or the Group of Schizophrenias.
        International Universities Press, New York1950
        • Kring A.M.
        • Germans M.K.
        Anhedonia.
        in: Kazdin A.E. Encyclopedia of Psychology. Oxford University Press, New York2000: 174-175
        • American Psychiatric Association
        Diagnostic and Statistical Manual of Mental Disorders, DSM-5.
        American Psychiatric Publishing, Washington, DC2013
        • Gard D.E.
        • Kring A.M.
        • Gard M.G.
        • Horan W.P.
        • Green M.F.
        Anhedonia in schizophrenia: Distinctions between anticipatory and consummatory pleasure.
        Schizophr Res. 2007; 93: 253-260
        • Vignapiano A.
        • Mucci A.
        • Ford J.
        • Montefusco V.
        • Plescia G.M.
        • Bucci P.
        • Galderisi S.
        Reward anticipation and trait anhedonia: An electrophysiological investigation in subjects with schizophrenia.
        Clin Neurophysiol. 2016; 127: 2149-2160
        • Murray G.K.
        • Clark L.
        • Corlett P.R.
        • Blackwell A.D.
        • Cools R.
        • Jones P.B.
        • et al.
        Incentive motivation in first-episode psychosis: A behavioural study.
        BMC Psychiatry. 2008; 8: 34
        • Schott B.H.
        • Minuzzi L.
        • Krebs R.M.
        • Elmenhorst D.
        • Lang M.
        • Winz O.H.
        • et al.
        Mesolimbic functional magnetic resonance imaging activations during reward anticipation correlate with reward-related ventral striatal dopamine release.
        J Neurosci. 2008; 28: 14311-14319
        • Leroy A.
        • Amad A.
        • D’Hondt F.
        • Pins D.
        • Jaafari N.
        • Thomas P.
        • Jardri R.
        Reward anticipation in schizophrenia: A coordinate-based meta-analysis.
        Schizophr Res. 2020; 218: 2-6
        • Wulff S.
        • Nielsen M.Ø.
        • Rostrup E.
        • Svarer C.
        • Jensen L.T.
        • Pinborg L.
        • Glenthøj B.Y.
        The relation between dopamine D2 receptor blockade and the brain reward system: A longitudinal study of first-episode schizophrenia patients.
        Psychol Med. 2020; 50: 220-228
        • Juckel G.
        • Schlagenhauf F.
        • Koslowski M.
        • Filonov D.
        • Wüstenberg T.
        • Villringer A.
        • et al.
        Dysfunction of ventral striatal reward prediction in schizophrenic patients treated with typical, not atypical, neuroleptics.
        Psychopharmacol (Berl). 2006; 187: 222-228
        • Nielsen M.Ø.
        • Rostrup E.
        • Broberg B.V.
        • Wulff S.
        • Glenthøj B.
        Negative symptoms and reward disturbances in schizophrenia before and after antipsychotic monotherapy.
        Clin EEG Neurosci. 2018; 49: 36-45
        • Vink M.
        • de Leeuw M.
        • Pouwels R.
        • van den Munkhof H.E.
        • Kahn R.S.
        • Hillegers M.
        Diminishing striatal activation across adolescent development during reward anticipation in offspring of schizophrenia patients.
        Schizophr Res. 2016; 170: 73-79
        • Gold J.M.
        • Waltz J.A.
        • Prentice K.J.
        • Morris S.E.
        • Heerey E.A.
        Reward processing in schizophrenia: A deficit in the representation of value.
        Schizophr Bull. 2008; 34: 835-847
        • Chase H.W.
        • Loriemi P.
        • Wensing T.
        • Eickhoff S.B.
        • Nickl-Jockschat T.
        Meta-analytic evidence for altered mesolimbic responses to reward in schizophrenia.
        Hum Brain Mapp. 2018; 39: 2917-2928
        • Fervaha G.
        • Takeuchi H.
        • Foussias G.
        • Hahn M.K.
        • Agid O.
        • Remington G.
        Achievement motivation in early schizophrenia: Relationship with symptoms, cognition and functional outcome.
        Early Interv Psychiatry. 2018; 12: 1038-1044
        • Green M.F.
        • Horan W.P.
        • Barch D.M.
        • Gold J.M.
        Effort-based decision making: A novel approach for assessing motivation in schizophrenia.
        Schizophr Bull. 2015; 41: 1035-1044
        • Wolf D.H.
        • Satterthwaite T.D.
        • Kantrowitz J.J.
        • Katchmar N.
        • Vandekar L.
        • Elliott M.A.
        • Ruparel K.
        Amotivation in schizophrenia: Integrated assessment with behavioral, clinical, and imaging measures.
        Schizophr Bull. 2014; 40: 1328-1337
        • Chang W.C.
        • Chu A.O.K.
        • Treadway M.T.
        • Strauss G.P.
        • Chan S.K.W.
        • Lee E.H.M.
        • et al.
        Effort-based decision-making impairment in patients with clinically stabilized first-episode psychosis and its relationship with amotivation and psychosocial functioning.
        Eur Neuropsychopharmacol. 2019; 29: 629-642
        • Pretus C.
        • Bergé D.
        • Guell X.
        • Pérez V.
        • Vilarroya Ó.
        Brain activity and connectivity differences in reward value discrimination during effort computation in schizophrenia.
        Eur Arch Psychiatry Clin Neurosci. 2021; 271: 647-659
        • Conn K.A.
        • Burne T.H.J.
        • Kesby J.P.
        Subcortical dopamine and cognition in schizophrenia: Looking beyond psychosis in preclinical models.
        Front Neurosci. 2020; 14: 542
        • Suetani S.
        • Baker A.
        • Garner K.
        • Cosgrove P.
        • Mackay-Sim M.
        • Siskind D.
        • et al.
        Impairments in goal-directed action and reversal learning in a proportion of individuals with psychosis.
        Cogn Affect Behav Neurosci. 2022; 22: 1390-1403
        • Morris R.W.
        • Quail S.
        • Griffiths K.R.
        • Green M.J.
        • Balleine B.W.
        Corticostriatal control of goal-directed action is impaired in schizophrenia.
        Biol Psychiatry. 2015; 77: 187-195
        • Morris R.W.
        • Cyrzon C.
        • Green M.J.
        • Le Pelley M.E.
        • Balleine B.W.
        Impairments in action–outcome learning in schizophrenia.
        Transl Psychiatry. 2018; 8: 54
        • Kesby J.P.
        • Eyles D.W.
        • McGrath J.J.
        • Scott J.G.
        Dopamine, psychosis and schizophrenia: The widening gap between basic and clinical neuroscience.
        Transl Psychiatry. 2018; 8: 30
        • Lex B.
        • Hauber W.
        The role of dopamine in the prelimbic cortex and the dorsomedial striatum in instrumental conditioning.
        Cereb Cortex. 2010; 20: 873-883
        • Howard C.D.
        • Li H.
        • Geddes C.E.
        • Jin X.
        Dynamic nigrostriatal dopamine biases action selection.
        Neuron. 2017; 93: 1436-1450.e8
        • Culbreth A.J.
        • Moran E.K.
        • Barch D.M.
        Effort-cost decision-making in psychosis and depression: Could a similar behavioral deficit arise from disparate psychological and neural mechanisms?.
        Psychol Med. 2018; 48: 889-904
        • Insel C.
        • Kastman E.K.
        • Glenn C.R.
        • Somerville L.H.
        Development of corticostriatal connectivity constrains goal-directed behavior during adolescence.
        Nat Commun. 2017; 8: 1605
        • Deserno L.
        • Huys Q.J.M.
        • Boehme R.
        • Buchert R.
        • Heinze H.J.
        • Grace A.A.
        • et al.
        Ventral striatal dopamine reflects behavioral and neural signatures of model-based control during sequential decision making.
        Proc Natl Acad Sci U S A. 2015; 112: 1595-1600
        • Insel C.
        • Charifson M.
        • Somerville L.H.
        Neurodevelopmental shifts in learned value transfer on cognitive control during adolescence.
        Dev Cogn Neurosci. 2019; 40: 100730
        • Li T.
        • Wang Q.
        • Zhang J.
        • Rolls E.T.
        • Yang W.
        • Palaniyappan L.
        • et al.
        Brain-wide analysis of functional connectivity in first-episode and chronic stages of schizophrenia.
        Schizophr Bull. 2017; 43: 436-448
        • Chopra S.
        • Francey S.M.
        • O’Donoghue B.
        • Sabaroedin K.
        • Arnatkeviciute A.
        • Cropley V.
        • et al.
        Functional connectivity in antipsychotic-treated and antipsychotic-naive patients with first-episode psychosis and low risk of self-harm or aggression: A secondary analysis of a randomized clinical trial.
        JAMA Psychiatry. 2021; 78: 994-1004
        • Betz L.T.
        • Brambilla P.
        • Ilankovic A.
        • Premkumar P.
        • Kim M.S.
        • Raffard S.
        • et al.
        Deciphering reward-based decision-making in schizophrenia: A meta-analysis and behavioral modeling of the Iowa Gambling Task.
        Schizophr Res. 2019; 204: 7-15
        • Reddy L.F.
        • Waltz J.A.
        • Green M.F.
        • Wynn J.K.
        • Horan W.P.
        Probabilistic reversal learning in schizophrenia: Stability of deficits and potential causal mechanisms.
        Schizophr Bull. 2016; 42: 942-951
        • Henquet C.
        • van Os J.
        • Pries L.K.
        • Rauschenberg C.
        • Delespaul P.
        • Kenis G.
        • et al.
        A replication study of JTC bias, genetic liability for psychosis and delusional ideation.
        Psychol Med. 2022; 52: 1777-1783
        • Ermakova A.O.
        • Gileadi N.
        • Knolle F.
        • Justicia A.
        • Anderson R.
        • Fletcher P.C.
        • et al.
        Cost evaluation during decision-making in patients at early stages of psychosis.
        Comput Psychiatr. 2019; 3: 18-39
        • Catalan A.
        • Tognin S.
        • Kempton M.J.
        • Stahl D.
        • Salazar de Pablo G.
        • Nelson B.
        • et al.
        Relationship between jumping to conclusions and clinical outcomes in people at clinical high-risk for psychosis.
        Psychol Med. 2020; 1–9
        • Koch K.
        • Schachtzabel C.
        • Wagner G.
        • Schikora J.
        • Schultz C.
        • Reichenbach J.R.
        • et al.
        Altered activation in association with reward-related trial-and-error learning in patients with schizophrenia.
        Neuroimage. 2010; 50: 223-232
        • Deserno L.
        • Boehme R.
        • Mathys C.
        • Katthagen T.
        • Kaminski J.
        • Stephan K.E.
        • et al.
        Volatility estimates increase choice switching and relate to prefrontal activity in schizophrenia.
        Biol Psychiatry Cogn Neurosci Neuroimaging. 2020; 5: 173-183