A Massive 7T fMRI Dataset to Bridge Neuroscience and Artificial Intelligence
Post by Andrew Vo
The takeaway
To understand the complex brain networks that underlie human sensory and cognitive behaviors, enormous amounts of high-quality imaging data are required. The introduction of such a dataset will be invaluable in studying processes such as vision or memory and will bridge the gap between cognitive neuroscience and artificial intelligence.
What's the science?
To successfully understand human brain function, we need to build comprehensive models of how information is processed by the brain. Such models require massive amounts of high-dimensional and context-specific data. However, most existing human brain imaging studies have been limited by small amounts of low-resolution data collected from varying numbers of individuals. This week in Nature Neuroscience, Allen et al. introduced the Natural Scenes Dataset (NSD), a publicly available brain imaging dataset of unprecedented scale and quality.
How did they do it?
The authors recruited eight human participants to contribute to the NSD. Each participant underwent whole-brain 7T functional magnetic resonance imaging (fMRI), during which their brain activity was measured as they viewed thousands of distinct natural scene images. 7T refers to the high magnetic field strength of the MRI scanner. Higher field strengths improve the signal-to-noise ratio and spatial resolution of the collected data, as compared to those data obtained at lower field strengths (consider that most hospital MRI scanners are only 1.5-3T). The participants collectively viewed over 70,000 richly annotated natural scenes across more than 300 scanning sessions held over the course of one year. To ensure participants remained attentive and engaged with the images, the authors simultaneously performed a continuous recognition task that involved indicating if a presented image was previously viewed. The authors carefully evaluated the data quality of the NSD and present initial analyses of the data.
What did they find?
The resulting NSD is the largest of its kind to date. High performance on the continuous recognition task indicated that participants were consistently engaged and attentive while viewing the many thousands of natural scene images. Inspection of the imaging data revealed that the signal-to-noise ratio and estimated brain responses across the brain remained stable across scanning sessions for each participant.
The authors demonstrated two initial applications of the NSD: First, they analyzed patterns of brain responses to the content of natural scenes and observed transformations of semantic representations along the ventral visual pathway. For example, brain patterns associated with people and animals were found in different parts of higher visual areas. Second, they applied machine learning techniques to build and train a deep convolutional neural network to predict brain activity in the brain’s visual areas. The large amount of data afforded by the NSD allowed their models to successfully predict brain activity more accurately than existing state-of-the-art models.
What's the impact?
This report introduces the NSD, a large-scale publicly available brain imaging dataset. The NSD is unique from other resources in terms of its massive scale (i.e., large amounts of data collected from individuals at ultra-high field strength), data quality, and novel analysis techniques. This sharable dataset has wide-ranging applications to the fields of cognitive science, neuroscience, artificial intelligence, and their intersection.
