Action selection in early stages of psychosis: an active inference approach
===========================================================================

* Franziska Knolle
* Elisabeth Sterner
* Michael Moutoussis
* Rick A. Adams
* Juliet D. Griffin
* Joost Haarsma
* Hilde Taverne
* Ian M. Goodyer
* Paul C. Fletcher
* Graham K. Murray

## Abstract

**Background:** To interact successfully with their environment, humans need to build a model to make sense of noisy and ambiguous inputs. An inaccurate model, as suggested to be the case for people with psychosis, disturbs optimal action selection. Recent computational models, such as active inference, have emphasized the importance of action selection, treating it as a key part of the inferential process. Based on an active inference framework, we sought to evaluate previous knowledge and belief precision in an action-based task, given that alterations in these parameters have been linked to the development of psychotic symptoms. We further sought to determine whether task performance and modelling parameters would be suitable for classification of patients and controls.

**Methods:** Twenty-three individuals with an at-risk mental state, 26 patients with first-episode psychosis and 31 controls completed a probabilistic task in which action choice (go/no-go) was dissociated from outcome valence (gain or loss). We evaluated group differences in performance and active inference model parameters and performed receiver operating characteristic (ROC) analyses to assess group classification.

**Results:** We found reduced overall performance in patients with psychosis. Active inference modelling revealed that patients showed increased forgetting, reduced confidence in policy selection and less optimal general choice behaviour, with poorer action–state associations. Importantly, ROC analysis showed fair-to-good classification performance for all groups, when combining modelling parameters and performance measures.

**Limitations:** The sample size is moderate.

**Conclusion:** Active inference modelling of this task provides further explanation for dysfunctional mechanisms underlying decision-making in psychosis and may be relevant for future research on the development of biomarkers for early identification of psychosis.

## Introduction

To make the most adaptive choices, the brain’s fundamental computational challenge is to integrate sensory data and prior knowledge while accounting for their uncertainties, as both sources are inconclusive by themselves.1,2 This process is formalized as Bayesian inference, in which an initial probabilistic expectation about the state of the environment (hereafter, the prior) is combined with the probability of the observed sensory data (its likelihood) to compute an updated prediction (the posterior), in which the contributions of the prior and the likelihood are weighted by their relative precisions.2,3 Effective action selection is characterized by maximizing rewards and minimizing losses through forming accurate beliefs. Thus, in addition to inferring the current state of the environment, the brain is also required to consider possible action outcomes to choose the most probable policy, given the organism’s expectations about its goals and about the likely state of the environment.

In psychosis, optimal action selection seems to be impaired, as alterations in value-based action selection (e.g., reinforcement learning)4,5 and in action–outcome (reward and punishment) learning can be observed.6–12 These behavioural and cognitive alterations often precede disease onset and, therefore, are a robust sign of the pathophysiology of the disorder.13,14 Computational models of decision-making allow the investigation of whether and how these impairments contribute to positive and negative symptoms. Reinforcement learning and active inference are 2 frameworks that use different algorithms to approximate how the brain optimizes action selection. Associative reinforcement learning models (such as model-free reinforcement learning) make the assumption that fairly simple associative updates (mainly based on reward presentation) are able to accommodate complex task structures, and that relatively simple update rules allow powerful performance, even when the associations are shaped by higher order contingencies.2,3

Active inference models, on the other hand, explicitly posit a more complex model of the structure of the task and use Bayesian inference, not just to infer hidden states of the world, but also to plan and select actions. Here, the primary goal of the agent is to minimize surprise (e.g., “I want to go outside and I do not want to be too hot”), which the agent achieves by acting to bring about sensory inputs that it expects (e.g., “If I wear shorts and a t-shirt, I will be nice and cool”) or, in Bayesian terms, by maximizing evidence for its assumptions about the environment (i.e., model evidence), given the context or state of the world (e.g., “It is sunny and people are walking around in short sleeves; ergo, I am expecting it to be warm, so shorts and t-shirt will keep me cool”).3,15 In other words, the brain predicts the consequences of an action based on both past experiences and the structure of the task, and will then choose the action expected to produce its most preferred outcomes.16 When an individual is unable to use sensory information to correct prior beliefs, or when their prior beliefs are inaccurate and the precision of sensory information is inaccurate, psychotic symptoms may arise, especially hallucinations.3,15,17,18 Not only sensory processing but also decision-making may be conceptualized as an inferential process, during which prior beliefs are the basis of inferring hidden states of the world.19–21

In a recent study involving individuals with an at-risk mental state (ARMS), patients with first-episode psychosis (FEP) and healthy controls, we investigated whether reinforcement learning, in particular, acts as an intermediate phenotype between genetic predispositions and the expressed clinical symptoms.22 We used an orthogonalized go/ no-go task that was designed to ensure that the reward and punishment outcomes of trials were dissociated from whether a go or no-go action was required.23 Using a reinforcement learning modelling algorithm, we found that reward and punishment sensitivity was reduced in individuals with ARMS and patients with FEP compared with controls. The reinforcement learning model of the go/no-go task allows the study of computational processes of the brain that mechanistically underlie and lead to specific task performances. Active inference, although more complex, has some potential advantages over existing reinforcement learning models of this task, as follows: active inference incorporates task structure (as a model-based reinforcement learning agent would); it treats apparent biases towards certain actions (e.g., no-go in negatively valanced states) as prior beliefs rather than as fixed action–selection biases, making them easier for agents to overcome; and active inference can update the confidence with which it chooses actions (i.e., its policy precision) and, thus, becomes less random in its choices as it learns more about the task.1,23–28 Indeed, psychotic symptoms may occur as a result of an imbalance between the precision of prior beliefs in relation to sensory evidence.3,17,29 Using the same task as our previous study,22 Adams and colleagues1 showed that an active inference model outperformed the reinforcement learning models in the better-performing individuals.

Cognitive dysfunction, such as dysfunction in decision-making, is a core feature of psychosis and has been found to predict poor functional and clinical treatment outcomes.30 Levels of cognitive impairment have been reported to be intermediate among at-risk individuals, compared with healthy controls and patients with schizophrenia, but without clear evidence for subsequent decline.14,31 Investigating cognitive impairments in decision-making using a modelling approach in at-risk individuals may, therefore, allow the identification of a potential biomarker of risk that improves early identification of psychosis and intervention.32,33

In this study, we aimed to investigate whether active inference parameters of the modelled orthogonalized go/no-go task differ between individuals with ARMS, patients with FEP and healthy controls, and whether they are linked to symptoms.23 Based on previous results using the reinforcement learning parameters,22 we hypothesized that patients with FEP would show reduced prior on policy precision, forgetting, optimism prior, and Pavlovian win and loss prior compared with controls and individuals with ARMS; individuals with ARMS take an intermediate position. We expected that Pavlovian win and loss prior would correlate with positive and negative symptoms in patients, and forgetting would correlate with general cognition across all participants. Furthermore, we sought to explore whether task performance and modelling parameters would be suitable for classification of patients and controls.

## Methods

### Participants

We included participants from the Neuroscience Clinical Adolescent and Adult Psychiatry Study Psychosis data set, which consists of 3 groups (aged 17–35 yr), including 31 controls, 23 individuals with ARMS and 26 patients with FEP. Participants were recruited from the wider population of Cambridgeshire in the United Kingdom. Patients with FEP were recruited from the Cambridge First Episode Psychosis service, CAMEO. Classification of individuals with ARMS was based on the Comprehensive Assessment for At Risk Mental States (CAARMS),34 as described by Morrison and colleagues.35 All individuals with ARMS met CAARMS criteria of attenuated psychotic symptoms. Controls were recruited through advertisement in Cambridgeshire and through existing research databases at the University of Cambridge. Montagnese and colleagues22 provide a detailed description of selection and classification.

### Psychological and clinical measures

Montagnese and colleagues22 provide a description of all measures assessed. The relevant measures for the present study are the matrix subscore of the Wechsler Abbreviated Scale of Intelligence (WASI), the CAARMS, the Schizotypal Personality Questionnaire (SPQ), the Positive and Negative Symptoms Scale (PANSS), the Cardiff Anomalous Perceptions Scale (CAPS40), the Peters et al. Delusions Inventory (PDI41) and the Mood and Feelings Questionnaire (MFQ), a subset of the Young People Questionnaire, to measure depressive symptoms.36–42

### Go/no-go task

All participants completed an orthogonalized go/no-go task, which allows for the investigation of learning of state–action contingencies (Figure 1). The task is described in detail elsewhere.22,23 In short, on each trial, 1 of 4 different fractal images is presented randomly, and participants must perform either a go (i.e., action invigoration) or a no-go (i.e., action inhibition) response. The response leads to a probabilistic outcome, namely a win (+£0.5), a loss (–£0.5) or no change (£0). Each of the 4 images represents a condition but participants do not know this. In the 2 reward conditions (i.e., positive valence), a win or no change are possible outcomes; in the 2 punishment conditions (i.e., negative valence), a loss or no change are possible outcomes. Outcomes are assigned probabilistically (80:20). In each pair of reward and punishment conditions, the decisions to be taken to maximize the overall win are opposite: “go to win” and “no-go to win” for the reward conditions, and “no-go to avoid losing” and “go to avoid losing” for the punishment conditions. ”Go to win” and “no-go to avoid losing” are considered Pavlovian-congruent conditions, and “no-go to win” and “go to avoid losing” are considered Pavlovian-incongruent conditions.

![Figure 1](http://jpn.ca/https://www.jpn.ca/content/jpn/48/1/E78/F1.medium.gif)

[Figure 1](http://jpn.ca/content/48/1/E78/F1)

Figure 1 
Schematic illustration of orthogonalized go/no-go task.22,23,43

### Active inference modelling

In applying computational modelling to psychological tasks, we aimed to identify and estimate parameters that relate to the underlying psychological processes engaged in successful task performance and, thereby, to characterize alterations in the processes by clinical group.

We used an active inference modelling approach of the orthogonalized go/no-go task, as described by Adams and colleagues.1 This approach uses a partially observable Markov decision process to model the action-dependent state transitions. The agent has to infer the state they are in, considering all actions and outcomes of the past (subject to forgetting), and simultaneously, to infer optimal actions given the current state and take into consideration their preferences.44,45 In our model, Pavlovian behaviour is explained on the basis of prior beliefs, namely that go is more likely to be the correct action given a rewarding (win) context, and no-go is more likely to be the correct action given a punishment context (avoid losing). The model we used thus contained the following parameters: 2 Pavlovian priors, 1 in the context of reward (*P(a* = go|context = win)*, hereafter, Pavlovian win prior), 1 in the context of punishment (*P(a* = no go|context = avoid losing)*, hereafter, Pavlovian loss prior); an overall prior that the context is one of reward (*P(context = win)*, hereafter, optimism prior); the precision of the preferences over outcomes, quantifying how strongly rewards are preferred over losses (hereafter, outcome sensitivity); the prior on policy precision, quantifying confidence in choosing (given one’s knowledge and preferences); and, finally, forgetting, assessing working memory.

Importantly, in the active inference model, parameters are estimated based on the complete cohort of all 3 groups to avoid false-positive biases that can occur in some types of hierarchical model fitting.46 This allowed us to investigate the relationship between model parameters and group classification. We estimated all parameters using approximate Bayesian inference, with the Laplace approximation to estimate free energy. For further analysis, we used the maximum of the posterior distribution for each parameter. We used the TAPAS toolbox, implemented in MATLAB ([http://www.translationalneuromodeling.org/tapas/](http://www.translationalneuromodeling.org/tapas/)).47,48

The same data were previously analyzed using an associative (reinforcement learning) model.22 These approaches model largely similar constructs. The active inference model represents a hypothesis that the Pavlovian guidance of behaviour encodes and then updates probabilistic expectations about environmental contingencies. Hence, the optimism and Pavlovian priors recast the Pavlovian and go bias of the reinforcement learning model, but allow the beliefs in question to be modified by experience. Reward and punishment sensitivity (i.e., inverse temperature) and lapse rate refer to the prior on policy precision and preferences over outcomes. In other words, the reinforcement learning model accounts for the difference between valences in terms of outcome sensitivity, whereas the active inference model accounts for this difference in terms of baseline beliefs about the world (i.e., prevalence priors). Moreover, forgetting is analogous to 1 minus the learning rate in reinforcement learning, but the learning rates in reinforcement learning account equally for learning and forgetting. In comparison, our active inference model applies perfect inference, but then forgets information the more a trial of 1 type is distant from the next. Finally, the stochasticity of behaviour can be regarded as fixed in the reinforcement learning model, but is more flexible in active inference, through updates of the policy precision. In addition, parameter estimation was realized differently in each model. Our previous work used a hierarchical approach and fitted participant groups separately to optimize individual parameter accuracy.22 Here, we used an approach more appropriate to detect correlations with clinical variables, reducing a bias at a slight expense of accuracy of individual parameters.

### Statistical analysis

We measured group differences in demographic information and clinical measures using the Pearson χ2 test, Welch *t* test and one-way analysis of variance (ANOVA). If the assumption of normality or homogeneity of variance were not met, we used the Wilcoxon rank-sum test or Kruskal–Wallis test, respectively.

We assessed group differences in performance between the 4 conditions (go to win, no-go to win, no-go to avoid losing, go to avoid losing) using the within-variables of valence (positive v. negative) and action (invigoration v. inhibition) in a mixed ANOVA design with subsequent Tukey post hoc tests.

We performed robust ANOVAs and post hoc tests based on trimmed means using the bootstrap method to compare active inference parameters between groups.

We used logistic regression to analyze the relationship between active inference parameters and group membership. We performed receiver operating characteristic (ROC) analyses, and assessed the area under the curve (AUC) to evaluate whether active inference parameters contribute to the classification of individuals. We defined the AUC thresholds for classification as excellent (0.90–1), good (0.80–0.89), fair (0.70–0.79), poor (0.60–0.69) or fail (0.50–0.59).49

Finally, we used heat maps reporting Pearson correlation analyses to link active inference parameters with clinical measures (i.e., the WASI matrix subscore, the CAARMS score, the SPQ score, the PANSS positive and negative symptom subscores, the CAPS score, the PDI score and the MFQ score2).

We conducted all statistical analyses in R, and visualized data using the ggplot2 package version 3.3.5.50,51 To compute the Levene test for homogeneity of variance, we used the car package version 3.0–11.52 To explore the assumption of normality, we performed the Shapiro–Wilk test using the pastecs package version 1.3.53 We conducted the Pearson χ2 test, Kruskal–Wallis test, Welch *t* test and Wilcoxon rank-sum test with the stats package version 4.0.54 We completed ANOVA analyses using the afex package version 1.0–1, and post hoc tests with the emmeans package version 1.5.3.55,56 We performed the robust ANOVAs and robust post hoc tests using the WRS2 package version 1.1–3,57 based on 20% trimmed means and 2000 bootstrap samples.

We implemented logistic regressions with the mlogit 1.1–1 package.58 We completed ROC and AUC analyses with the pROC 1.18.0 package.59 Finally, we performed and visualized correlation analyses with the ggcorrplot package version 0.1.3.60

### Ethics approval

The study was approved by the Cambridgeshire 3 National Health Service research ethics committee. All participants gave written informed consent in accordance with the Declaration of Helsinki.

## Results

Table 1 describes participants, including medication status, who were included in all analyses.

View this table:
[Table 1](http://jpn.ca/content/48/1/E78/T1)

Table 1 
Summary demographic information and clinical measures by group

### Behavioural performance

Figure 2A depicts the learning rates of each group for each condition, ordered along the axes of action and valence. In Figure 2B, accuracy values for valence and action are presented in boxplots for each group. Individual performances at 10 and 30 trials are presented in Figure S1 in Appendix 1, available at [www.jpn.ca/lookup/doi/10.1503/jpn.220141/tab-related-content](http://www.jpn.ca/lookup/doi/10.1503/jpn.220141/tab-related-content).

![Figure 2:](http://jpn.ca/https://www.jpn.ca/content/jpn/48/1/E78/F2.medium.gif)

[Figure 2:](http://jpn.ca/content/48/1/E78/F2)

Figure 2: 
Overview of task performance. (A) Learning rate by group and trial type. Patients with first-episode psychosis performed worse (relative to controls) in the “avoid losing” conditions (lower plots). (B) Behavioural performance. Box plots show the medians as horizontal bars, means as small red dots, group means as large red dots and interquartile ranges as whiskers; significant group differences of the Tukey post hoc tests are shown (**p* < 0.05, ***p* < 0.01, \***|*p* < 0.001). ARMS = individuals with at-risk mental state, CON = control group, FEP = patients with first-episode psychosis.

Based on these results, we conducted a 3 × 2 × 2 mixed ANOVA with group (control v. individuals with ARMS v. patients with FEP) as a between-group variable, and valence (positive v. negative) and action (invigoration v. inhibition) as within-group variables. We observed a significant main effect for group (*F*2,77 = 8.02, *p* < 0.001, η2*p*= 0.17) and action (*F*1,77 = 9.30, *p* = 0.003, η2*p* = 0.11) but not for valence (*F*1,77 = 0.15, *p* = 0.695, η2 = 0.00). Although there was no interaction between group and valence (*F*2,77 = 1.79, *p* = 0.174, η2*p* = 0.04) or group and action (*F*2,77 = 0.09, *p* = 0.917, η2 = 0.00), there was a significant interaction between action and valence (*F*1,77 = 41.65, *p* < 0.001, η2*p* = 0.35). This is the Pavlovian effect, as will be discussed below. The interaction of group, valence and action was not significant (*F*2,77 = 0.42, *p* = 0.660, η2 = 0.01).

Tukey post hoc analysis found that controls (mean difference 13.98, 95% confidence interval [CI] 4.79 to 23.16) and individuals with ARMS (mean difference 13.59, 95% CI 3.70 to 23.48) were significantly more accurate than patients with FEP. There was no significant difference in accuracy between controls and individuals with ARMS (mean difference 0.39, 95% CI −9.12 to 9.89).

Regarding the interaction between action and valence, Tukey post hoc analysis showed that participants were more accurate when performing a Pavlovian congruent response than a Pavlovian incongruent response (go to win v. go to avoid losing: mean difference −15.3, 95% CI −20.0 to −10.7); no-go to avoid losing v. no-go to win: mean difference 14.3, 95% CI 8.5 to 20.1).

We also explored the interaction between the within-group variables and group (Figure 1B). For valence, Tukey post hoc analysis showed that controls (mean difference 16.54, 95% CI 5.54 to 27.53) and individuals with ARMS (mean difference 16.28, 95% CI 4.45 to 28.11) were significantly more accurate in punishment trials than patients with FEP, but not in reward trials. For action, performance did not differ between go and no-go trials across groups.

### Computational modelling

We performed 1-way robust ANOVAs and robust post hoc tests based on the trimmed means using the bootstrap method to compare active inference parameters between groups. We found significant differences for forgetting, as well as for the fit measures of free energy and maximum likelihood (Table 2 and Figure 3A). To investigate potential effects of gender or antipsychotic medication on active inference parameters, we conducted additional control analyses (Appendix 1, Figure S2, Table S3, Figure S4 and Table S5).

![Figure 3:](http://jpn.ca/https://www.jpn.ca/content/jpn/48/1/E78/F3.medium.gif)

[Figure 3:](http://jpn.ca/content/48/1/E78/F3)

Figure 3: 
Group comparisons of active inference parameters and model fit from robust analysis of variance based on trimmed means using the bootstrap method. Significant results from robust post hoc analyses are shown (**p* < 0.05, ***p* < 0.01). Horizontal bars of boxplots mark the median and whiskers indicate the interquartile range. ARMS = patients with at-risk mental state, CON = control group, FEP = patients with first-episode psychosis.

View this table:
[Table 2](http://jpn.ca/content/48/1/E78/T2)

Table 2 
Summary of descriptive statistics for active inference parameters by group

### Classification based on model parameters

The AUC from the ROC analyses — representing the overall classification performance based on logistic regression using only the active inference modelling parameters (i.e., Pavlovian win prior, Pavlovian loss prior, optimism prior, outcome sensitivity, forgetting, prior on policy precision, free energy) — are presented in Figure 4A. Classification performances differed depending on group comparison. The controls were differentiated from individuals with ARMS with an overall poor performance (AUC 0.6690, specificity 0.65, sensitivity 0.61, accuracy 0.63). Importantly, the patients with FEP were differentiated from controls with a fair performance (AUC 0.7917, specificity 0.83, sensitivity 0.68, accuracy 0.75), and from individuals with ARMS with a good performance (AUC 0.8188, specificity 0.67, sensitivity 0.83, accuracy 0.74), indicating that active inference model parameters contributed significantly to group classification. Regression results are in Appendix 1, Figure S6.

![Figure 4:](http://jpn.ca/https://www.jpn.ca/content/jpn/48/1/E78/F4.medium.gif)

[Figure 4:](http://jpn.ca/content/48/1/E78/F4)

Figure 4: 
Receiver operating characteristic curves for group comparisons based on classification models using (A) modelling parameters, (B) performance measures and (C) a combination of modelling parameters and performance measures. ARMS = individuals with at-risk mental state, AUC = area under the curve, CON = control group, FEP = patients with first-episode psychosis.

### Classification based on performance

The AUC from the ROC analyses — representing the overall classification performance based on logistic regression using only the performance in the 4 conditions (go to win, no-go to win, no-go to avoid losing, go to avoid losing) — are presented in Figure 4B. Classification performances differed depending on group comparison. Using performance, the classification of controls from individuals with ARMS failed (AUC 0.5694, specificity 0.48, sensitivity 0.58, accuracy 0.54). Importantly, patients with FEP were differentiated from controls (AUC 0.8078, specificity 0.75, sensitivity 0.71, accuracy 0.73), and from individuals with ARMS with a good performance (AUC 0.7953, specificity 0.79, sensitivity 0.70, accuracy 0.74). Regression results are in Appendix 1, Figure S7.

### Classification based on model parameters and performance

The AUC from the ROC analyses — representing the overall classification performance based on logistic regression using the performance in the 4 conditions (go to win, no-go to win, no-go to avoid losing, go to avoid losing) and the active inference modelling parameters (Pavlovian win prior, Pavlovian loss prior, optimism prior, outcome sensitivity, forgetting, prior on policy precision, free energy) — are presented in Figure 4C. Classification performances differed depending on group comparison. Importantly, patients with FEP were differentiated from controls with a good performance (AUC 0.8804, specificity 0.79, sensitivity 0.81, accuracy 0.80), as well as from individuals with ARMS (AUC 0.8877, specificity 0.75, sensitivity 0.83, accuracy 0.79). In contrast to the performance-only classification’s failure to differentiate these 2 groups, in this analysis, controls were differentiated from individuals with ARMS with an overall fair performance (AUC 0.7419, specificity 0.70, sensitivity 0.65, accuracy 0.67). Regression results are in Appendix 1, Figure S8.

To identify which parameter specifically led to a classification improvement, we conducted a leave-1-out analysis by systematically excluding parameters from the logistic regression and subsequent ROC analysis. Leaving out the Pavlovian win prior, the free energy parameter and the task performance in the “no-go to avoid losing” condition produced a poor classification performance (Appendix 1, Figure S9). Excluding other active inference parameters generated similar AUC values, but were still above the threshold for a fair classification performance.

### Relationship between active inference parameters and clinical parameters

The correlation analysis between modelling parameters and clinical scores did not show any significant results when correcting for multiple comparisons (uncorrected results are in Appendix 1, Figure S10).

## Discussion

We investigated whether Bayesian measures of decision-making that underly action selection differed among individuals with ARMS and those with FEP, compared with healthy controls, and whether these measures were associated with symptoms. We applied an active inference model to an orthogonalized go/no-go task for individuals with ARMS, patients with FEP and healthy controls.1,22,23 Our results showed that patients with FEP had significantly worse performances in the punishment condition than individuals with ARMS and controls. Furthermore, patients with FEP had an overall increased forgetting parameter relative to the other groups. Investigating the Bayesian model measures, we found that patients with FEP had significantly lower levels of free energy and maximum likelihood than controls, indicating that these patients were less Bayesian-optimal in their inferences than the other groups.

Interestingly, we found a group-level trend toward lower prior on policy precision, with post hoc tests revealing a difference between individuals with ARMS compared with controls. This is consistent with current theories of an imbalance of prior precision compared with sensory likelihood underlying the psychopathology of symptoms in psychosis.29,61,62 It may also reconcile the findings of Moutoussis and colleagues63 with those of Ermakova and colleagues.11 The former study, which included patients with longer-term psychosis, found that inconsistent choosing might be associated with lower policy precision, but this was not observed in the latter study, which focused on individuals with ARMS and early psychosis. However, our data showed that there were 8 individuals, 5 of whom were in the FEP group, with the prior on policy precision above the group-specific standard deviation, suggesting greater variability with respect to this measure, especially in the FEP group.

The task used in this study disentangled the type of action (go v. no-go) from the valence of the action outcome (reward v. punishment). Consistent with previous research using this task,23,25 participants generally performed better in the Pavlovian-congruent conditions (go to win, no-go to avoid losing) compared with the incongruent conditions (go to avoid losing, no-go to win). Previous research has discussed how impairments in action selection are an intermediate phenotype between neurobiological or genetic substrates and expressed clinical symptoms, which is why we would expect these alterations in decision-making to also be present in individuals with increased clinical risk of developing psychosis; however, findings are inconsistent.13,64,65 Our findings do not confirm this hypothesis. In our sample, we identified poorer performance among patients with FEP compared with controls but also compared with individuals with ARMS, mainly in the punishment condition. In a recent study that administered a probabilistic learning task to patients with early or persistent psychosis, Suetani and colleagues66 reported that patients with early psychosis were less likely to adapt their behaviour after a loss compared with controls, which is indicative of deficits in punishment learning and is similar to our findings.

Using the active inference model parameters, we identified differences between patients with FEP and controls, as well as individuals with ARMS. Interestingly, we found that patients with FEP were less likely to achieve a Bayesian-optimal outcome selection, indicated by lower free energy and maximum likelihood. Importantly, we found a reduced prior on policy precision among patients with FEP compared with controls and individuals with ARMS. It has been suggested that psychosis is represented by an imbalance of the precision of the prior relative to the precision of sensory information.3,15,67–69 This interaction may be dependent on the hierarchical level, with increased precision of the prior at higher hierarchical levels and decreased precision at lower hierarchical levels.29,62,70,71 Our results showed lower prior precision and higher forgetting among patients with FEP, indicating that they showed deficits in identifying and possibly maintaining the associations between cue and outcome, as well as their respective probabilities, which may be linked to altered beliefs about environmental volatilities.70,72,73

Adams and colleagues1 reported that prior precision is negatively correlated with D2/3 receptor availability in the limbic striatum, linking greater D2/3 receptor availability to lower precision. Findings of D2/3 receptor availability in psychosis, mainly in the striatum and the thalamus, have been linked to medication status, indicating that antipsychotic-naïve individuals have the same D2/3 receptor availability as controls.74,75 Some studies, however, have also found an increased availability before or without treatment.76 Adams and colleagues1 reported that lower D2/3 receptor availability can be cognitively advantageous, as higher tonic dopamine activity may be associated with higher prior precision. We speculate that patients with FEP with lower prior precision may have more disordered dopaminergic transmission, but this will need further investigation.

We applied the active inference model to the whole sample to reduce the likelihood of false-positive findings.43,46 Although this procedure may be slightly less sensitive to group-specific differences, fitting the posterior distribution over all participant performances allows conservative use of the parameter output for classification analyses. Computational modelling of neurocognitive tasks sheds light on the neuropsychological processes underlying the behaviour, which may be linked to the psychopathology of the disorder.2,68 Using model parameters in classification algorithms combines 2 approaches necessary to improve strategies of identifying individuals with psychosis early and precisely.77,78

The detection of potential biomarkers for early identification of individuals at risk is one of the goals of current research efforts. Cognitive impairments are particularly interesting owing to their prodromal onset and their impact on functional outcomes.14,30,79–81 Using ROC analysis, we found that the specific expression of modelling parameters, as well as individual performance measures, allowed a good classification of patients with FEP, with significant differentiation from controls and individuals with ARMS. This indicates that differences in the neuropsychological processes underlying the performance in the go/no-go task are relevant for the psychopathology of psychosis.9,13,19 Importantly, however, combining modelling parameters and performance measures in the ROC analysis improved the identification of all group associations by at least 8%. This improvement was especially important for the distinction of controls and individuals with ARMS, leading to a fair classification performance. This finding is highly relevant for future research on biomarkers for early identification of psychosis, and should be validated in larger testing samples. An additional, explorative leave-1-out analysis indicated that the combination of all model parameters may have led to the classification improvement, rather than any single parameter.

### Limitations

Our study was limited by the number of trials, as not all learning curves reached a plateau by the end of the task. Future studies should consider running the task with more trials to ensure that all participants have learned sufficiently. Our study was limited by the sample size. However, the active inference model was fitted across all 81 participants, which is a sufficiently high number, and produced excellent model convergence. To further increase the confidence in our analysis, we used robust ANOVA with resampling, which yielded reliable effects with medium effect sizes. The current sample size did not allow an investigation of effect of antipsychotic medication on modelling parameters. In the future, alternative designs such as crossover studies or randomized controlled trials could clarify medication effects.

## Conclusion

We found that, among patients with FEP, deficits in probabilistic decision-making in an orthogonalized go/no-go task were linked to increased forgetting, reduced prior precision and less optimal general choice behaviour, with poorer punishment learning. Reduced prior precision in FEP may be linked to alterations in tonic striatal dopaminergic activity, which is associated with D2/3 receptor availability. Our results support findings of previous studies and provide further mechanistic insights about how altered cognitive parameters may lead to dysfunctional decision-making in psychosis.

## Acknowledgement

The authors thank all participants for their time and engagement.

## Footnotes

*   * These authors contributed equally to this work.

*   **Contributors:** Franziska Knolle, Juliet Griffin, Joost Haarsma, Ian Goodyer, Paul Fletcher and Graham Murray contributed to the conception and design of the work. Juliet Griffin, Joost Haarsma, Hilde Taverne, Ian Goodyer and Graham Murray contributed to data acquisition. Franziska Knolle, Elisabeth Sterner, Michael Moutoussis, Rick Adams and Graham Murray contributed to data analysis and interpretation. Franziska Knolle and Elisabeth Sterner drafted the manuscript. All authors revised it critically for important intellectual content, gave final approval of the version to be published and agreed to be accountable for all aspects of the work. Franziska Knolle and Elisabeth Sterner share first authorship.

*   **Data availability:** Data are available in the Cambridge University data repository.

*   **Competing interests:** Franziska Knolle reports funding from the European Union’s Horizon 2020 fund. Joost Haarsma reports travel support from the American College of Neuropsychopharmacology. Paul Fletcher reports consulting fees from Ninja Theory. All competing interests are outside the submitted work.

*   **Funding:** This work was supported by the Neuroscience in Psychiatry Network, a strategic award from the Wellcome Trust to the University of Cambridge and University College London (095844/Z/11/Z).

*   Received August 9, 2022.
*   Revision received November 11, 2022.
*   Accepted November 28, 2022.

This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY-NC-ND 4.0) licence, which permits use, distribution and reproduction in any medium, provided that the original publication is properly cited, the use is noncommercial (i.e., research or educational use), and no modifications or adaptations are made. See: [https://creativecommons.org/licenses/by-nc-nd/4.0/](https://creativecommons.org/licenses/by-nc-nd/4.0/)

## References

1.  Adams RA, Moutoussis M, Nour MM, et al. Variability in action selection relates to striatal dopamine 2/3 receptor availability in humans: a PET neuroimaging study using reinforcement learning and active inference models. Cereb Cortex 2020;30:3573–89.
    
    

2.  Adams RA, Huys QJM, Roiser JP. Computational psychiatry: towards a mathematically informed understanding of mental illness. J Neurol Neurosurg & Psychiatry 2016;87:53–63.
    
    [Abstract/FREE Full Text](http://jpn.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiam5ucCI7czo1OiJyZXNpZCI7czo3OiI4Ny8xLzUzIjtzOjQ6ImF0b20iO3M6MTg6Ii9qcG4vNDgvMS9FNzguYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

3.  Sterzer P, Adams RA, Fletcher P, et al. The predictive coding account of psychosis. Biol Psychiatry 2018;84:634–43.
    
    [CrossRef](http://jpn.ca/lookup/external-ref?access_num=10.1016/j.biopsych.2018.05.015&link_type=DOI) 
    
    [PubMed](http://jpn.ca/lookup/external-ref?access_num=30007575&link_type=MED&atom=%2Fjpn%2F48%2F1%2FE78.atom) 

4.  Waltz JA, Wilson RC, Albrecht MA, et al. Differential Effects of psychotic illness on directed and random exploration. Comput Psychiatr 2020;4:18–39.
    
    

5.  Gold JM, Strauss GP, Waltz JA, et al. Negative symptoms of schizophrenia are associated with abnormal effort-cost computations. Biol Psychiatry 2013;74:130–6.
    
    [CrossRef](http://jpn.ca/lookup/external-ref?access_num=10.1016/j.biopsych.2012.12.022&link_type=DOI) 
    
    [PubMed](http://jpn.ca/lookup/external-ref?access_num=23394903&link_type=MED&atom=%2Fjpn%2F48%2F1%2FE78.atom) 
    
    [Web of Science](http://jpn.ca/lookup/external-ref?access_num=000321108800010&link_type=ISI) 

6.  Deserno L, Boehme R, Heinz A, et al. Reinforcement learning and dopamine in schizophrenia: dimensions of symptoms or specific features of a disease group? Front Psychiatry 2013;4:172.
    
    [CrossRef](http://jpn.ca/lookup/external-ref?access_num=10.3389/fpsyt.2013.00172&link_type=DOI) 
    
    [PubMed](http://jpn.ca/lookup/external-ref?access_num=24391603&link_type=MED&atom=%2Fjpn%2F48%2F1%2FE78.atom) 

7.  Gold JM, Waltz JA, Prentice KJ, et al. Reward processing in schizophrenia: a deficit in the representation of value. Schizophr Bull 2008;34:835–47.
    
    [CrossRef](http://jpn.ca/lookup/external-ref?access_num=10.1093/schbul/sbn068&link_type=DOI) 
    
    [PubMed](http://jpn.ca/lookup/external-ref?access_num=18591195&link_type=MED&atom=%2Fjpn%2F48%2F1%2FE78.atom) 
    
    [Web of Science](http://jpn.ca/lookup/external-ref?access_num=000259322600007&link_type=ISI) 

8.  Morris RW, Cyrzon C, Green MJ, et al. Impairments in action–outcome learning in schizophrenia. Transl Psychiatry 2018;8:54.
    
    

9.  Kesby JP, Murray GK, Knolle F. Neural circuitry of salience and reward processing in psychosis. Biol Psychiatry Glob Open Sci 2023;3:33–46.
    
    

10. Ermakova AO, Knolle F, Justicia A, et al. Abnormal reward prediction-error signalling in antipsychotic naive individuals with first-episode psychosis or clinical risk for psychosis. Neuropsychopharmacology 2018;43:1691–9.
    
    [CrossRef](http://jpn.ca/lookup/external-ref?access_num=10.1038/s41386-018-0056-2&link_type=DOI) 

11. Ermakova AO, Gileadi N, Knolle F, et al. Cost evaluation during decision-making in patients at early stages of psychosis. Comput Psychiatry 2019;3:18–39.
    
    

12. Maia TV, Frank MJ. An integrative perspective on the role of dopamine in schizophrenia. Biol Psychiatry 2017;81:52–66.
    
    [CrossRef](http://jpn.ca/lookup/external-ref?access_num=10.1016/j.biopsych.2016.05.021&link_type=DOI) 

13. Strauss GP, Datta R, Armstrong W, et al. Reinforcement learning abnormalities in the attenuated psychosis syndrome and first episode psychosis. Eur Neuropsychopharmacol 2021;47:11–9.
    
    [PubMed](http://jpn.ca/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fjpn%2F48%2F1%2FE78.atom) 

14. Fusar-Poli P, Deste G, Smieskova R, et al. Cognitive functioning in prodromal psychosis: a meta-analysis. Arch Gen Psychiatry 2012;69: 562–71.
    
    [CrossRef](http://jpn.ca/lookup/external-ref?access_num=10.1001/archgenpsychiatry.2011.1592&link_type=DOI) 
    
    [PubMed](http://jpn.ca/lookup/external-ref?access_num=22664547&link_type=MED&atom=%2Fjpn%2F48%2F1%2FE78.atom) 
    
    [Web of Science](http://jpn.ca/lookup/external-ref?access_num=000304771000003&link_type=ISI) 

15. Adams R, Stephan K, Brown H, et al. The computational anatomy of psychosis. Front Psychiatry 2013;4:47.
    
    [CrossRef](http://jpn.ca/lookup/external-ref?access_num=10.3389/fpsyt.2013.00047&link_type=DOI) 
    
    [PubMed](http://jpn.ca/lookup/external-ref?access_num=23750138&link_type=MED&atom=%2Fjpn%2F48%2F1%2FE78.atom) 

16. Smith R, Friston KJ, Whyte CJ. A step-by-step tutorial on active inference and its application to empirical data. J Math Psychol 2022; 107:102632.
    
    

17. Benrimoh D, Parr T, Adams RA, et al. Hallucinations both in and out of context: an active inference account. PLoS One 2019;14: e0212379.
    
    

18. Benrimoh D, Parr T, Vincent P, et al. Active inference and auditory hallucinations. Comput Psychiatry 2018;2:183.
    
    

19. Sterzer P, Voss M, Schlagenhauf F, et al. Decision-making in schizophrenia: a predictive-coding perspective. Neuroimage 2019; 190:133–43.
    
    

20. Friston K. The free-energy principle: a unified brain theory? Nat Rev Neurosci 2010;11:127–38.
    
    [CrossRef](http://jpn.ca/lookup/external-ref?access_num=10.1038/nrn2787&link_type=DOI) 
    
    [PubMed](http://jpn.ca/lookup/external-ref?access_num=20068583&link_type=MED&atom=%2Fjpn%2F48%2F1%2FE78.atom) 
    
    [Web of Science](http://jpn.ca/lookup/external-ref?access_num=000273784600012&link_type=ISI) 

21. Botvinick M, Toussaint M. Planning as inference. Trends Cogn Sci 2012;16:485–8.
    
    [CrossRef](http://jpn.ca/lookup/external-ref?access_num=10.1016/j.tics.2012.08.006&link_type=DOI) 
    
    [PubMed](http://jpn.ca/lookup/external-ref?access_num=22940577&link_type=MED&atom=%2Fjpn%2F48%2F1%2FE78.atom) 
    
    [Web of Science](http://jpn.ca/lookup/external-ref?access_num=000309848700001&link_type=ISI) 

22. Montagnese M, Knolle F, Haarsma J, et al. Reinforcement learning as an intermediate phenotype in psychosis? Deficits sensitive to illness stage but not associated with polygenic risk of schizophrenia in the general population. Schizophr Res 2020;222:389–96.
    
    

23. Guitart-Masip M, Huys QJM, Fuentemilla L, et al. Go and no-go learning in reward and punishment: interactions between affect and effect. Neuroimage 2012;62:154–66.
    
    [CrossRef](http://jpn.ca/lookup/external-ref?access_num=10.1016/j.neuroimage.2012.04.024&link_type=DOI) 
    
    [PubMed](http://jpn.ca/lookup/external-ref?access_num=22548809&link_type=MED&atom=%2Fjpn%2F48%2F1%2FE78.atom) 
    
    [Web of Science](http://jpn.ca/lookup/external-ref?access_num=000305859300018&link_type=ISI) 

24. de Boer JN, Linszen MMJ, de Vries J, et al. Auditory hallucinations, top-down processing and language perception: a general population study. Psychol Med 2019;49:2772–80.
    
    

25. Huys QJM, Cools R, Golzer M, et al. Disentangling the roles of approach, activation and valence in instrumental and pavlovian responding. PLoS Comput Biol 2011;7:e1002028.
    
    [CrossRef](http://jpn.ca/lookup/external-ref?access_num=10.1371/journal.pcbi.1002028&link_type=DOI) 
    
    [PubMed](http://jpn.ca/lookup/external-ref?access_num=21556131&link_type=MED&atom=%2Fjpn%2F48%2F1%2FE78.atom) 

26. Swart JC, Frobose MI, Cook JL, et al. Catecholaminergic challenge uncovers distinct Pavlovian and instrumental mechanisms of motivated (in)action. Elife 2017;6:e22169.
    
    [CrossRef](http://jpn.ca/lookup/external-ref?access_num=10.7554/eLife.22169&link_type=DOI) 
    
    [PubMed](http://jpn.ca/lookup/external-ref?access_num=28504638&link_type=MED&atom=%2Fjpn%2F48%2F1%2FE78.atom) 

27. Chowdhury R, Guitart-Masip M, Lambert C, et al. Structural integrity of the substantia nigra and subthalamic nucleus predicts flexibility of instrumental learning in older-age individuals. Neurobiol Aging 2013;34:2261–70.
    
    

28. Cavanagh JF, Eisenberg I, Guitart-Masip M, et al. Frontal theta overrides pavlovian learning biases. J Neurosci 2013;33:8541–8.
    
    [Abstract/FREE Full Text](http://jpn.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6Njoiam5ldXJvIjtzOjU6InJlc2lkIjtzOjEwOiIzMy8xOS84NTQxIjtzOjQ6ImF0b20iO3M6MTg6Ii9qcG4vNDgvMS9FNzguYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

29. Weilnhammer V, Rod L, Eckert A-L, et al. Psychotic experiences in schizophrenia and sensitivity to sensory evidence. Schizophr Bull 2020; 46:927–36.
    
    

30. Allott K, Liu P, Proffitt T-M, et al. Cognition at illness onset as a predictor of later functional outcome in early psychosis: systematic review and methodological critique. Schizophr Res 2011;125:221–35.
    
    [CrossRef](http://jpn.ca/lookup/external-ref?access_num=10.1016/j.schres.2010.11.001&link_type=DOI) 
    
    [PubMed](http://jpn.ca/lookup/external-ref?access_num=21111577&link_type=MED&atom=%2Fjpn%2F48%2F1%2FE78.atom) 
    
    [Web of Science](http://jpn.ca/lookup/external-ref?access_num=000288304000016&link_type=ISI) 

31. Bora E, Murray RM. Meta-analysis of cognitive deficits in ultra-high risk to psychosis and first-episode psychosis: do the cognitive deficits progress over, or after, the onset of psychosis? Schizophr Bull 2014;40:744–55.
    
    [CrossRef](http://jpn.ca/lookup/external-ref?access_num=10.1093/schbul/sbt085&link_type=DOI) 
    
    [PubMed](http://jpn.ca/lookup/external-ref?access_num=23770934&link_type=MED&atom=%2Fjpn%2F48%2F1%2FE78.atom) 
    
    [Web of Science](http://jpn.ca/lookup/external-ref?access_num=000338130300007&link_type=ISI) 

32. Fusar-Poli P. Predicting psychosis. Arch Gen Psychiatry 2012;69:220.
    
    [CrossRef](http://jpn.ca/lookup/external-ref?access_num=10.1001/archgenpsychiatry.2011.1472&link_type=DOI) 
    
    [PubMed](http://jpn.ca/lookup/external-ref?access_num=22393215&link_type=MED&atom=%2Fjpn%2F48%2F1%2FE78.atom) 
    
    [Web of Science](http://jpn.ca/lookup/external-ref?access_num=000301066200002&link_type=ISI) 

33. Velthorst E, Zinberg J, Addington J, et al. Potentially important periods of change in the development of social and role functioning in youth at clinical high risk for psychosis. Dev Psychopathol 2018;30:39–47.
    
    

34. Yung AR, Yuen HP, McGorry PD, et al. Mapping the onset of psychosis: the comprehensive assessment of at risk mental states. Schizophr Res 2005;39:964–71.
    
    [CrossRef](http://jpn.ca/lookup/external-ref?access_num=doi:10.1016/S0920-9964(03)80090-7&link_type=DOI) 

35. Morrison AP, Stewart SLK, French P, et al. Early detection and intervention evaluation for people at risk of psychosis: multisite randomised controlled trial. Br Med J 2012;344:e2233.
    
    [Abstract/FREE Full Text](http://jpn.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjE3OiIzNDQvYXByMDVfMS9lMjIzMyI7czo0OiJhdG9tIjtzOjE4OiIvanBuLzQ4LzEvRTc4LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

36. Wechsler D. Wechsler abbreviated scale of intelligence. New York: The Psychological Corporation; 1999.
    
    

37. Yung AR, Yuen HP, McGorry PD, et al. Mapping the onset of psychosis: the comprehensive assessment of at-risk mental states. Aust N Z J Psychiatry 2005;39:964–71.
    
    [CrossRef](http://jpn.ca/lookup/external-ref?access_num=10.1111/j.1440-1614.2005.01714.x&link_type=DOI) 
    
    [PubMed](http://jpn.ca/lookup/external-ref?access_num=16343296&link_type=MED&atom=%2Fjpn%2F48%2F1%2FE78.atom) 
    
    [Web of Science](http://jpn.ca/lookup/external-ref?access_num=000233205400003&link_type=ISI) 

38. Raine A. The SPQ: a scale for the assessment of schizotypal personality based on DSM-III-R criteria. Schizophr Bull 1991;17:555–64.
    
    [CrossRef](http://jpn.ca/lookup/external-ref?access_num=10.1093/schbul/17.4.555&link_type=DOI) 
    
    [PubMed](http://jpn.ca/lookup/external-ref?access_num=1805349&link_type=MED&atom=%2Fjpn%2F48%2F1%2FE78.atom) 
    
    [Web of Science](http://jpn.ca/lookup/external-ref?access_num=A1991GX26000003&link_type=ISI) 

39. Kay SR, Fiszbein A, Opler LA. The positive and negative syndrome scale (PANSS) for schizophrenia. Schizophr Bull 1987;13:261–76.
    
    [CrossRef](http://jpn.ca/lookup/external-ref?access_num=10.1093/schbul/13.2.261&link_type=DOI) 
    
    [PubMed](http://jpn.ca/lookup/external-ref?access_num=3616518&link_type=MED&atom=%2Fjpn%2F48%2F1%2FE78.atom) 
    
    [Web of Science](http://jpn.ca/lookup/external-ref?access_num=A1987H977500007&link_type=ISI) 

40. Bell V, Halligan PW, Ellis HD. The Cardiff Anomalous Perceptions Scale (CAPS): a new validated measure of anomalous perceptual experience. Schizophr Bull 2006;32:366–77.
    
    [CrossRef](http://jpn.ca/lookup/external-ref?access_num=10.1093/schbul/sbj014&link_type=DOI) 
    
    [PubMed](http://jpn.ca/lookup/external-ref?access_num=16237200&link_type=MED&atom=%2Fjpn%2F48%2F1%2FE78.atom) 
    
    [Web of Science](http://jpn.ca/lookup/external-ref?access_num=000236106600031&link_type=ISI) 

41. Peters E, Joseph S, Day S, et al. Measuring delusional ideation: the 21-item Peters et al. Delusions Inventory (PDI). Schizophr Bull 2004; 30:1005–22.
    
    [CrossRef](http://jpn.ca/lookup/external-ref?access_num=10.1093/oxfordjournals.schbul.a007116&link_type=DOI) 
    
    [PubMed](http://jpn.ca/lookup/external-ref?access_num=15954204&link_type=MED&atom=%2Fjpn%2F48%2F1%2FE78.atom) 
    
    [Web of Science](http://jpn.ca/lookup/external-ref?access_num=000227916000029&link_type=ISI) 

42. Costello EJ, Angold A. Scales to assess child and adolescent depression: checklists, screens, and nets. J Am Acad Child Adolesc Psychiatry 1988;27:726–37.
    
    [CrossRef](http://jpn.ca/lookup/external-ref?access_num=10.1097/00004583-198811000-00011&link_type=DOI) 
    
    [PubMed](http://jpn.ca/lookup/external-ref?access_num=3058677&link_type=MED&atom=%2Fjpn%2F48%2F1%2FE78.atom) 
    
    [Web of Science](http://jpn.ca/lookup/external-ref?access_num=A1988Q943900011&link_type=ISI) 

43. Moutoussis M, Rutledge RB, Prabhu G, et al. Neural activity and fundamental learning, motivated by monetary loss and reward, are intact in mild to moderate major depressive disorder. PLoS One 2018;13:e0201451.
    
    

44. Friston K. Active inference and free energy. Behav Brain Sci 2013; 36:212.
    
    [CrossRef](http://jpn.ca/lookup/external-ref?access_num=10.1017/S0140525X12002142&link_type=DOI) 
    
    [PubMed](http://jpn.ca/lookup/external-ref?access_num=23663424&link_type=MED&atom=%2Fjpn%2F48%2F1%2FE78.atom) 

45. Friston K, Schwartenbeck P, Fitzgerald T, et al. The anatomy of choice: active inference and agency. Front Hum Neurosci 2013;7:598.
    
    [CrossRef](http://jpn.ca/lookup/external-ref?access_num=10.3389/fnhum.2013.00598&link_type=DOI) 
    
    [PubMed](http://jpn.ca/lookup/external-ref?access_num=24093015&link_type=MED&atom=%2Fjpn%2F48%2F1%2FE78.atom) 

46. Moutoussis M, Hopkins AK, Dolan RJ. Hypotheses about the relationship of cognition with psychopathology should be tested by embedding them into empirical priors. Front Psychol 2018;9:2504.
    
    

47. Mathys C, Daunizeau J, Friston KJ, et al. A Bayesian foundation for individual learning under uncertainty. Front Hum Neurosci 2011; 5:39.
    
    [CrossRef](http://jpn.ca/lookup/external-ref?access_num=10.3389/fnhum.2011.00039&link_type=DOI) 
    
    [PubMed](http://jpn.ca/lookup/external-ref?access_num=21629826&link_type=MED&atom=%2Fjpn%2F48%2F1%2FE78.atom) 

48. Frässle S, Aponte EA, Bollmann S, et al. TAPAS: an open-source software package for translational neuromodeling and computational psychiatry. Front Psychiatry 2021;12:680811.
    
    [CrossRef](http://jpn.ca/lookup/external-ref?access_num=10.3389/fpsyt.2021.680811&link_type=DOI) 

49. Safari S, Baratloo A, Elfil M, et al. Evidence based emergency medicine; part 5 receiver operating curve and area under the curve. Emergency 2016;4:111.
    
    

50. Rstudio Team. RStudio: integrated development for R. Boston: Rstudio Team, PBC; 2020. Available: [http://www.rstudio.com/](http://www.rstudio.com/) (accessed 2022 Oct. 26).
    
    

51. Wickham H. ggplot2: elegant graphics for data analysis. New York: Springer; 2016.
    
    

52. Fox J, Weisberg S. An R companion to applied regression. Los Angeles: Sage; 2018.
    
    

53. Grosjean P, Ibanez F, Etienne M. Pastecs: Package for analysis of space-time ecological series. R package version 1.3. 21. 2018.
    
    

54. Team RCR: A language and environment for statistical computing. R Foundation for Statistical Computing: Vienna, Austria; 2020.
    
    

55. Singmann H, Bolker B, Westfall J, et al. afex: Analysis of factorial experiments. R Packag. version 0.13–145. 2015.
    
    

56. Lenth R. emmeans: estimated marginal means, aka least-squares means, v1. 5.1. Vienna R Core Team; 2020.
    
    

57. Mair P, Wilcox R. Robust statistical methods in R using the WRS2 package. Behav Res Methods 2020;52:464–88.
    
    [CrossRef](http://jpn.ca/lookup/external-ref?access_num=10.3758/s13428-019-01246-w&link_type=DOI) 

58. Croissant Y. Estimation of random utility models in R: the mlogit package. J Statistical Software 2020;95:1–41.
    
    

59. Robin X, Turck N, Hainard A, et al. pROC: an open-source package for R and S+ to analyze and compare ROC curves. BMC Bioinformatics 2011;12:77.
    
    [CrossRef](http://jpn.ca/lookup/external-ref?access_num=10.1186/1471-2105-12-77&link_type=DOI) 
    
    [PubMed](http://jpn.ca/lookup/external-ref?access_num=21414208&link_type=MED&atom=%2Fjpn%2F48%2F1%2FE78.atom) 

60. Kassambara A, Kassambara MA. Package ‘ggcorrplot’. R Package. version 0.1 3. 2019.
    
    

61. Powers AR, Mathys C, Corlett PR. Pavlovian conditioning–induced hallucinations result from overweighting of perceptual priors. Science 2017;357:596–600.
    
    [Abstract/FREE Full Text](http://jpn.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6Mzoic2NpIjtzOjU6InJlc2lkIjtzOjEyOiIzNTcvNjM1MS81OTYiO3M6NDoiYXRvbSI7czoxODoiL2pwbi80OC8xL0U3OC5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

62. Haarsma J, Knolle F, Griffin JD, et al. Influence of prior beliefs on perception in early psychosis: Effects of illness stage and hierarchical level of belief. J Abnorm Psychol 2020;129:581–98.
    
    

63. Moutoussis M, Bentall RP, El-Deredy W, et al. Bayesian modelling of Jumping-to-Conclusions bias in delusional patients. Cogn Neuropsychiatry 2011;16:422–47.
    
    

64. Millman ZB, Gallagher K, Demro C, et al. Evidence of reward system dysfunction in youth at clinical high-risk for psychosis from two event-related fMRI paradigms. Schizophr Res 2020;226:111–9.
    
    

65. Waltz J, Demro C, Schiffman J, et al. Reinforcement learning performance and risk for psychosis in youth. J Nerv Ment Dis 2015; 203:919.
    
    

66. Suetani S, Baker A, Garner K, et al. Impairments in goal-directed action and reversal learning in a proportion of individuals with psychosis: evidence for differential phenotypes in early and persistent psychosis. medRxiv 2021
    
    

67. Corlett PR, Frith CD, Fletcher PC. From drugs to deprivation: a Bayesian framework for understanding models of psychosis. Psychopharmacology 2009;206:515–30.
    
    [CrossRef](http://jpn.ca/lookup/external-ref?access_num=10.1007/s00213-009-1561-0&link_type=DOI) 
    
    [PubMed](http://jpn.ca/lookup/external-ref?access_num=19475401&link_type=MED&atom=%2Fjpn%2F48%2F1%2FE78.atom) 

68. Heinz A, Murray GK, Schlagenhauf F, et al. Towards a unifying cognitive, neurophysiological, and computational neuroscience account of schizophrenia. Schizophr Bull 2019;45:1092–1100.
    
    

69. Haarsma J, Kok P, Browning M. The promise of layer-specific neuroimaging for testing predictive coding theories of psychosis. Schizophr Res 2020;254:68–76.
    
    

70. Haarsma J, Fletcher PC, Griffin JD, et al. Precision weighting of cortical unsigned prediction error signals benefits learning, is mediated by dopamine, and is impaired in psychosis. Mol Psychiatry 2020;26:5320–33.
    
    

71. Teufel C, Subramaniam N, Dobler V, et al. Shift toward prior knowledge confers a perceptual advantage in early psychosis and psychosis-prone healthy individuals. Proc Natl Acad Sci U S A 2015; 112:13401–6.
    
    [Abstract/FREE Full Text](http://jpn.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoicG5hcyI7czo1OiJyZXNpZCI7czoxMjoiMTEyLzQzLzEzNDAxIjtzOjQ6ImF0b20iO3M6MTg6Ii9qcG4vNDgvMS9FNzguYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

72. Culbreth AJ, Westbrook A, Xu Z, et al. Intact ventral striatal prediction error signaling in medicated schizophrenia patients. Biol Psychiatry Cogn Neurosci Neuroimaging 2016;1:474–83.
    
    

73. Schlagenhauf F, Huys QJM, Deserno L, et al. Striatal dysfunction during reversal learning in unmedicated schizophrenia patients. Neuroimage 2014;89:171–80.
    
    [CrossRef](http://jpn.ca/lookup/external-ref?access_num=10.1016/j.neuroimage.2013.11.034&link_type=DOI) 
    
    [PubMed](http://jpn.ca/lookup/external-ref?access_num=24291614&link_type=MED&atom=%2Fjpn%2F48%2F1%2FE78.atom) 

74. Chen KC, Yang YK, Howes OD, et al. Striatal dopamine D2/3 receptors in medication-naïve schizophrenia: an [123I] IBZM SPECT study. Psychol Med 2022:52;3251–9.
    
    

75. Rajji TK, Mulsant BH, Nakajima S, et al. Cognition and dopamine D2 receptor availability in the striatum in older patients with schizophrenia. Am J Geriatr Psychiatry 2017;25:1–10.
    
    

76. Veselinovic T, Vernaleken I, Janouscheck H, et al. The role of striatal dopamine D2/3 receptors in cognitive performance in drug-free patients with schizophrenia. Psychopharmacology (Berl) 2018;235: 2221–32.
    
    

77. Wiecki TV, Poland J, Frank MJ. Model-based cognitive neuroscience approaches to computational psychiatry: clustering and classification. Clin Psychol Sci 2015;3378–99.
    
    

78. Huys QJM, Maia TV, Frank MJ. Computational psychiatry as a bridge from neuroscience to clinical applications. Nat Neurosci 2016; 19:404–13.
    
    [CrossRef](http://jpn.ca/lookup/external-ref?access_num=10.1038/nn.4238&link_type=DOI) 
    
    [PubMed](http://jpn.ca/lookup/external-ref?access_num=26906507&link_type=MED&atom=%2Fjpn%2F48%2F1%2FE78.atom) 

79. Fett AKJ, Velthorst E, Reichenberg, et al. Long-term changes in cognitive functioning in individuals with psychotic disorders: findings from the suffolk county mental health project. JAMA Psychiatry 2020;77:387–96.
    
    

80. Savla GN, Vella L, Armstrong CC, et al. Deficits in domains of social cognition in schizophrenia: a meta-analysis of the empirical evidence. Schizophr Bull 2013;39:979–92.
    
    [CrossRef](http://jpn.ca/lookup/external-ref?access_num=10.1093/schbul/sbs080&link_type=DOI) 
    
    [PubMed](http://jpn.ca/lookup/external-ref?access_num=22949733&link_type=MED&atom=%2Fjpn%2F48%2F1%2FE78.atom) 
    
    [Web of Science](http://jpn.ca/lookup/external-ref?access_num=000323972700010&link_type=ISI) 

81. Guo JY, Niendam TA, Auther AM, et al. Predicting psychosis risk using a specific measure of cognitive control: a 12-month longitudinal study. Psychol Med 2020;50:2230–9.