ReviewTest-retest reliabilities of resting-state FMRI measurements in human brain functional connectomics: A systems neuroscience perspective
Introduction
Connectomics has rapidly become a revolution in basic brain research following the introduction of the key concept of the brain connectome, which conceptualizes the connection of the entire brain at different scales as a complex and dynamic system (Sporns et al., 2005, Sporns, 2013). The macro-scale brain structural connectome has been mapped out as a graph with 50–1000 nodes based on structural magnetic resonance imaging (MRI) (He et al., 2007) or diffusion tensor imaging (DTI) (Hagmann et al., 2008, Gong et al., 2009). Brain dynamics are shaped by the brain connectome structure and topology (Deco et al., 2011, Bullmore and Sporns, 2009, Bullmore and Sporns, 2012) and can be measured with resting-state functional MRI (RFMRI) with different spatial (millimeters) and temporal (seconds) resolutions (Biswal et al., 1995) as well as use of graph theory (Salvador et al., 2005, Achard et al., 2006). At this macro-scale, the functional connectivity of the entire brain connectome or brain graph (Bullmore and Bassett, 2011) has been termed the ‘functional connectome’ (Biswal et al., 2010, Zuo et al., 2012, Kelly et al., 2012), which, conceptually, is another term for the functional outcomes of the brain connectome.
Over the last two decades, macro-scale functional connectomics (i.e., the functional connectomes revealed with RFMRI) has rapidly become a powerful tool in the study of human brain function and associations with mind, behavior and disease (Craddock et al., 2013, Castellanos et al., 2013). For example, the default mode or default network (DMN) of the human brain represents a remarkable and successful indication of discovery science with functional connectomics (Buckner, 2012). This network originally referred to a set of brain regions including the posterior cingulate, hippocampus, medial prefrontal and inferior parietal cortex, which are functionally connected (Greicius et al., 2003) and reliably deactivates during most externally focused tasks (Shulman et al., 1997) but exhibits elevated metabolism during internal cognitive processes (Raichle et al., 2001). The function of the DMN has been related to episodic memory (Buckner et al., 2005), mind wandering (Mason et al., 2007) and various traits (e.g., Adelstein et al., 2011, personality). Furthermore, DMN alterations have been widely observed in neurodegenerative diseases such as Alzheimer's disease (AD) (Greicius et al., 2004) and psychiatric disorders (Greicius, 2008, Broyd et al., 2009, Fornito and Bullmore, 2010). Between-group differences or inter-individual variability in functional connectomics have greatly enriched our knowledge of specific cortical areas (e.g., (Leech and Sharp, 2014), the posterior cingulate cortex) and elucidated the meaningful and stable properties of the human brain functional connectivity (Buckner et al., 2013).
Although functional connectomes have demonstrated features that are temporally stable or exhibit statistically ignorable intra-individual variability, the temporal dynamics of these connectomes have recently received substantial attention (Hutchison et al., 2013). There are both neural and non-neural factors that likely contribute to the dynamic changes in resting-state functional connectivity. Although exploratory, emerging evidence suggests that dynamic RSFC patterns indicate the intrinsic functional architecture of the human brain (Deco and Corbetta, 2011) as it relates to normal cognition (Albert et al., 2009, Bassett et al., 2011, Mantzaris et al., 2013), behavior (Fox et al., 2007, Hesselmann et al., 2008, Sadaghiani et al., 2010) and clinical diseases (e.g., Jones et al., 2012, AD). Meanwhile, the intra-individual variability of resting-state functional connectivity can be partly attributed to various non-neural factors including scan conditions (Yan et al., 2009), head motion (Power et al., 2012, Van Dijk et al., 2012, Satterthwaite et al., 2012, Satterthwaite et al., 2013, Yan et al., 2013a), physiological noise (Birn et al., 2008, Chang et al., 2009, Chang and Glover, 2009a, Chang and Glover, 2009b), and data analysis/standardization strategies (Yan et al., 2013b). In summary, factors that can affect the intra-individual variability of RSFC patterns raise concerns regarding the test-retest reliabilities of functional metrics of the brain connectome with RFMRI.
High test-retest reliability requires both low intra-individual and high inter-individual variability (Barnhart et al., 2007). Low intra-individual variability indicates a high stability across different points in time, and high inter-individual variability implies highly differentiable measures across subjects. High test-retest reliability is particularly important for the development of biomarker-based clinical tests for early detection, timely interventions and diagnoses of brain disorders, especially psychiatric disorders, which currently lack a gold standard biological definition (Kapur et al., 2012). Surprisingly, however, the importance of test-retest reliability has been overlooked in functional connectomics until several recent studies on the test-retest reliability of RFMRI were published (Shehzad et al., 2009, Zuo et al., 2010a, Zuo et al., 2010b). As mentioned, RFMRI offers both high temporal and spatial resolutions to examine whole brain activity in vivo. A common spatial unit of RFMRI measurements is called the volumetric element (voxel) of several millimeters. While previous studies on functional connectomes conducted with large-scale parcellation provided great insight into brain network organization (Bullmore and Sporns, 2009, Bullmore and Sporns, 2012), regional variations in local functional homogeneity (Zang et al., 2004, Zuo et al., 2013) suggested that defining a node based on a large structural region and building an edge between a pair of nodes in a brain graph can be problematic. Simply averaging the voxel-wise time series in a large region ignores the fact that the strength of functional homogeneity within the region is typically low and highly variable across spatial locations (Jiang et al., 2014), leading to difficulties in interpreting the mean time series and derived functional metrics in relation to raw time series. This highlights the importance of voxel-wise assessments on the intrinsic functional architecture.
The present work will survey common voxel-wise metrics used in functional connectomics and their test-retest reliabilities. We also aim to provide recommendations on use of these functional metrics for characterizing the brain connectome at a systems neuroscience level by summarizing the voxel-wise reliability maps of seven functional networks of a large-scale functional brain parcellation. Fig. 1 demonstrates a flowchart of the overall analytic strategies. Specifically, we first introduce the RFMRI measurements of human functional connectomes at the voxel level (Fig. 1A–D) and then briefly review various voxel-wise functional metrics (Fig. 1D), the test-retest reliabilities of which have been examined previously (Fig. 1E). Finally, these reliability maps are summarized into a large-scale functional parcellation of the human brain as defined in (Yeo et al., 2011) (Fig. 1F) by comparing the distribution of voxel-wise test-retest reliability values (Fig. 1G), the proportion of reliable voxels (Fig. 1H) and the ratio of reliable voxels (Fig. 1I) across the seven functional networks.
Section snippets
Functional metrics of brain connectomes with RFMRI
The basal metabolism of the human brain measured during rest state (i.e., eyes closed, awake, and no specific cognitive task) represents 20% of the total body energy consumption, which highlights the importance of studying resting state brain function (Raichle, 2006, Raichle and Mintun, 2006). RFMRI was initially proposed to study low-frequency spontaneous neural activity in the motor system (Biswal et al., 1995). Since then, this technology has been widely used to study human brain function
Test-retest reliability in functional connectomics
We have briefly reviewed the high-resolution (i.e., voxel-wise) metrics of the human brain function. As mentioned, our aim with this review is to provide a systematic overview on the test-retest reliability of these voxel-wise functional metrics and further a reference on uses of these metrics to study the human functional connectome markers in brain disorders (Singh and Rose, 2009). In this section, the test-retest reliability of common functional metrics will be summarized at the level of
Further considerations and future directions
As noted, both within- and between-subject variability can affect the level of test-retest reliability. Any factor that changes within- or between-subject variability thus possibly influences test-retest reliability. Meanwhile, the findings revealed by the meta-summary analysis implied that there are remarkable differences in test-retest reliability across different functional metrics and functional networks. It becomes a crucial point in developing a highly reliable method to investigate how
Conclusions and recommendations
The overall spatial pattern of common RFMRI measurements can be reliably detected across different times and large-scale networks. The regional variations of test-retest reliability of various RFMRI metrics are reflected at the network level across the common seven large-scale neural networks, among which, the high-order associative networks have the highest reliability, whereas the limbic network exhibits the lowest reliability. The voxel-wise metrics of functional connectomes can exhibit
Funding
Dr. Xi-Nian Zuo acknowledges the funding support from the Hundred Talents Program, the Key Research Program (KSZD-EW-TZ-002) of the Chinese Academy of Sciences, the Major Joint Fund for International Cooperation and Exchange of the National Natural Science Foundation (81220108014). Dr. Xiu-Xia Xing's research is supported by the Fundamental Development Program on Mathematics and Statistics from Beijing University of Technology and the Natural Science Foundation of China (81201153). RFMRI data
Conflict of interest statement
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Acknowledgments
Dr. Xi-Nian Zuo is indebted to Yu-Feng Zang, Michael Peter Milham and F. Xavier Castellanos, who introduced him to open neuroscience with RFMRI and have provided endless supports. Without their encouragement and comments, this review would not have been possible. We thank Zarrar Shehzad and Chao-Gan Yan for sharing their reliability maps, Donna Dierker for discussions about the design and implementation of surface-based cortical analyses with the RFMRI data from the HCP Q3 release, Yong He and
References (232)
- et al.
The resting human brain and motor learning
Curr. Biol.
(2009) - et al.
The brain activity map project and the challenge of functional connectomics
Neuron
(2012) Modelling with independent components
Neuroimage
(2012)- et al.
Multi-level bootstrap analysis of stable clusters in resting-state fMRI
Neuroimage
(2010) - et al.
Alzheimer's disease neuroimaging. I. Spatiotemporal linear mixed effects modeling for the mass-univariate analysis of longitudinal neuroimage data
Neuroimage
(2013) The role of physiological noise in resting-state functional connectivity
Neuroimage
(2012)- et al.
The effect of scan length on the reliability of resting-state fMRI connectivity estimates
Neuroimage
(2013) - et al.
The respiration response function: the temporal dynamics of fMRI signal fluctuations related to changes in respiration
Neuroimage
(2008) - et al.
Spatially constrained hierarchical parcellation of the brain with resting-state fMRI
Neuroimage
(2013) - et al.
Test-retest reliability of resting-state connectivity network characteristics using fMRI and graph theoretical measures
Neuroimage
(2012)
Reliable quantification of BOLD fMRI cerebrovascular reactivity despite poor breath-hold performance
Neuroimage
Removing motion and physiological artifacts from intrinsic BOLD fluctuations using short echo data
Neuroimage
Default-mode brain dysfunction in mental disorders: a systematic review
Neurosci. Biobehav. Rev.
The serendipitous discovery of the brain's default network
Neuroimage
Topological organization of the human brain functional connectome across the lifespan
Dev. Cogn. Neurosci.
Clinical applications of the functional connectome
Neuroimage
Influence of heart rate on the BOLD signal: the cardiac response function
Neuroimage
Effects of model-based physiological noise correction on default mode network anti-correlations and correlations
Neuroimage
Relationship between respiration, end-tidal CO2, and BOLD signals in resting-state fMRI
Neuroimage
Linear mixed-effects modeling approach to FMRI group analysis
Neuroimage
Hierarchical clustering to measure connectivity in fMRI resting-state data
Magn. Reson. Imaging
AFNI: what a long strange trip it's been
Neuroimage
Reproducibility of graph metrics of human brain functional networks
Neuroimage
A predictive network model of cerebral cortical connectivity based on a distance rule
Neuron
FreeSurfer
Neuroimage
Intrinsic fluctuations within cortical systems account for intertrial variability in human behavior
Neuron
Network discovery with DCM
Neuroimage
Moment-to-moment brain signal variability: a next frontier in human brain mapping?
Neurosci. Biobehav. Rev.
The minimal preprocessing pipelines for the Human Connectome Project
Neuroimage
Accurate and robust brain image alignment using boundary-based registration
Neuroimage
One-year test-retest reliability of intrinsic connectivity network fMRI in older adults
Neuroimage
Functional connectivity in resting-state fMRI: is linear correlation sufficient?
Neuroimage
Dynamic functional connectivity: promise, issues, and interpretations
Neuroimage
FSL
Neuroimage
Brainnetome: a new -ome to understand the brain and its disorders
Neuroimage
Brainnetome-wide association studies in schizophrenia: the advances and future
Neurosci. Biobehav. Rev.
A resilient, low-frequency, small-world human brain functional network with highly connected association cortical hubs
J. Neurosci.
Personality is reflected in the brain's intrinsic functional architecture
PLoS One
Challenges and opportunities in mining neuroscience data
Science
Nanotools for neuroscience and brain activity mapping
ACS Nano
Neuroscience. the brain activity map
Science
Tracking whole-brain connectivity dynamics in the resting state
Cereb. Cortex
Decreased interhemispheric functional connectivity in autism
Cereb. Cortex
Reproducibility of single-subject functional connectivity measurements
AJNR Am. J. Neuroradiol.
Complexity of low-frequency blood oxygen level-dependent fluctuations covaries with local connectivity
Hum. Brain Map.
An overview on assessing agreement with continuous measurements
J. Biopharm. Stat.
Dynamic reconfiguration of human brain networks during learning
Proc. Natl. Acad. Sci. U. S. A.
Investigations into resting-state connectivity using independent component analysis
Philos. Trans. R. Soc. Lond. B Biol. Sci.
Alzheimer's Disease Neuroimaging I. Statistical analysis of longitudinal neuroimage data with linear mixed effects models
Neuroimage
Functional connectivity in the motor cortex of resting human brain using echo-planar MRI
Magn. Reson. Med.
Cited by (512)
Aberrant interhemispheric structural and functional connectivity within whole brain in schizophrenia
2024, Schizophrenia ResearchShared and distinctive dysconnectivity patterns underlying pure generalized anxiety disorder (GAD) and comorbid GAD and depressive symptoms
2024, Journal of Psychiatric ResearchTest-Retest Reliability of Functional Connectivity in Adolescents With Depression
2024, Biological Psychiatry: Cognitive Neuroscience and NeuroimagingPre-COVID brain network topology prospectively predicts social anxiety alterations during the COVID-19 pandemic
2023, Neurobiology of Stress