Timing and Reward in Perceptual Decision-Making

Post by Cody Walters 

What’s the science?

Decision-making often takes place in dynamic and uncertain environments, yet it remains poorly understood how this uncertainty affects time-sensitive decision-making strategies. This week in The Journal of Neuroscience, Shinn and colleagues, used a novel model to identify the cognitive mechanisms that explain behaviour during a perceptual decision-making task under temporally uncertain conditions.

How did they do it?

The authors trained two male rhesus macaques (monkeys) to perform a color-matching task. The animals fixated on a screen and were presented with a square grid (i.e., the ‘sample’) of randomly arranged blue and green pixels. The animals then had to indicate whether there were more blue or green pixels in the image by saccading (i.e., moving their eyes) to the corresponding color-coded target on the left or right of the sample in order to receive a juice reward. While the animals fixated on the screen prior to the sample being displayed, the targets indicated which of the two colors, if correctly selected, would result in a larger reward for that trial. Additionally, there was a non-informative ‘presample’ grid of blue and green pixels that was displayed for either 400 or 800 milliseconds preceding the onset of the sample.

To model evidence accumulation, the authors used drift-diffusion models (DDMs). A DDM captures how perceptual information is integrated over time and ultimately results in a choice being made between one of two alternatives. DDMs fundamentally involve a drift rate (how quickly the decision variable is accumulated) and decision boundary (which, when reached, triggers a response). In addition to a traditional DDM, the authors also used an extended version that they refer to as a generalized drift-diffusion model (GDDM). A GDDM incorporates features such as time-varying evidence accumulation (a ‘leak’ parameter), an urgency signal to discourage prolonged deliberation (through either a ‘collapsing bounds’ or ‘linear gain’ parameter), and reward bias (through either an ‘initial bias’ parameter that shifts the starting point of the decision variable, a ‘time-dependent bias’ parameter that accelerates the decision variable toward the direction of the larger reward option, or a ‘mapping error’ parameter that accounts for movement errors). Furthermore, a delay parameter was introduced to the urgency signals to model the uncertainty surrounding the stimulus onset.

What did they find?

The authors compared two DDM models in order to determine which provided a better fit to the monkeys’ choice and reaction time data. They found that the GDDM, relative to a reward-biased DDM (a traditional DDM with initial bias and time-dependent bias parameters), more accurately captured the choice and reaction time data and provided a better fit to the errors resulting from reward bias (where the monkeys would choose the large-reward option when the evidence favored the small-reward option). These findings suggested that the GDDM was needed in order to more closely examine the mechanisms influencing evidence accumulation under conditions of temporal uncertainty.

The authors then fit multiple GDDMs (with different parameter combinations) to the choice and reaction time data obtained from the two monkeys. They found that models with time-varying urgency outperformed models with a constant urgency signal (such as the traditional DDM), suggesting that whatever strategy the animals used to adapt to the temporal uncertainty of the stimulus onset was being captured by the time-varying urgency parameter. As for reward parameters, they found that a combination of initial bias (an integration-dependent mechanism) with mapping error (an integration-independent mechanism) provided the best fit to the experimental data. To address the extent to which the leak parameter and the time-varying urgency signal functionally overlapped, the authors then removed the leak parameter from the GDDMs. They found that GDDMs with a time-varying urgency parameter but without a leak parameter experienced significant impairment in performance, thus suggesting that leaky evidence accumulation improves the model fits by discounting earlier, more uncertain sensory evidence through a separate mechanism from the time-varying urgency signal.

Next, the authors investigated whether within-session changes in the temporal context would influence the urgency signal using a modified version of the task. Importantly, this task design ensured that 20% of trials in both the short and long presample blocks had a 0.75 second presample, which allowed the authors to assess the extent to which temporal context affected the urgency signal. The authors found that the best-fit GDDM obtained from the first task fit the monkey data from the modified task exceptionally well, with the urgency parameter alone accounting for most of the variation in the data. These findings show that the monkeys’ behavioral strategy in different temporal contexts is explained in large part by the timing of the urgency signal. Finally, the authors showed that modulation of the urgency signal (i.e., the decision boundary collapse delay) alone is sufficient to influence a speed-accuracy tradeoff, thus demonstrating an overall role for the urgency signal in time-dependent decision-making strategies.  

What’s the impact?

These findings demonstrate that (1) a dynamic urgency signal provides a flexible mechanism for incorporating sensory evidence into the decision-making process under temporal uncertainty and (2) that integration-dependent (initial bias and time-dependent bias) and integration-independent (mapping error) mechanisms are both required to explain reward bias. Overall, these data suggest that time is an important factor affecting how subjects integrate information and make decisions. Further, this study outlines a promising new model for exploring the cognitive mechanisms underlying perceptual decision-making.

Confluence of timing and reward biases in perceptual decision-making dynamics. The Journal of Neuroscience (2020). Access the original scientific publication here.