Reliable intrinsic connectivity networks: Test–retest evaluation using ICA and dual regression approach
Introduction
Functional connectivity of resting-state fMRI data is rapidly emerging as a highly efficient and powerful tool for in vivo mapping of neural circuitry in the human brain. Essentially, resting state functional connectivity (RSFC) approaches detect coherent patterns of low-frequency (< 0.1 Hz) fluctuations in the resting state BOLD signal, referred to as “intrinsic connectivity networks” (ICNs). Despite a burgeoning literature demonstrating the utility of RSFC approaches (Fox and Raichle, 2007), researchers continue to struggle with defining computationally efficient and reliable approaches for identifying and characterizing ICNs.
Inspired by a seminal demonstration of RSFC within the motor system (Biswal et al., 1995), seed-based correlation represents the dominant approach for studying RSFC. This method detects ICNs by identifying voxels whose timeseries significantly correlate with the mean timeseries of voxels within an a priori seed region of interest (ROI). In addition to generating highly detailed maps of complex functional systems (Margulies et al., 2007, Zhang et al., 2008, Di Martino et al., 2008, Roy et al., 2009, Nioche et al., 2009), seed-based correlation analyses are commonly used to identify abnormalities in ICNs related to psychopathology (Greicius, 2008, Broyd et al., 2009), as well as relationships between RSFC measures and individual differences in behavioral (Kelly et al., 2008, Fox et al., 2007) and trait characteristics (Di Martino et al., 2009). Overall, this mass-univariate approach has proven to be a powerful, efficient, and reliable tool for neuroimaging (Shehzad et al., 2009).
Seed-based analytic approaches are not without limitation, however. Most notably, the ICNs derived with seed-based correlation are highly dependent upon choice of a seed ROI. Further, the seed ROI timeseries includes indeterminate noise, in contrast to the idealized task design matrix used in task-based analyses. While several preprocessing strategies commonly are employed to remove noise (e.g., regressing out the global signal), such corrections can affect the data (e.g., inducing artifactual negative correlations) (Murphy et al., 2009). Finally, as a general linear model (GLM)-based univariate method, seed-based correlation disregards relationships among multiple voxels (i.e., it only analyzes the relationship between the seed timeseries and one other voxel at a time).
In light of these issues, neuroimaging researchers have adopted a multivariate signal processing method known as independent component analysis (ICA) to explore the spatial-temporal properties of resting state fMRI data (Kiviniemi, 2003, van de Ven et al., 2004, Beckmann et al., 2005). In theory, without any explicit a priori knowledge, ICA aims to separate spatially (spatial ICA: sICA) or temporally (temporal ICA: tICA) independent patterns from their linearly mixed BOLD signals via maximization of mutual independence among components (Stone et al., 1999). Generally, sICA is the more appropriate choice for analyzing resting state fMRI data given the small number of time points included in most fMRI datasets. Thus, we also used sICA in this paper. For simplicity, in the remainder of this paper, we refer to sICA as ICA.
ICA offers several potential advantages over seed-based correlation. First, as noted above, ICA is a multivariate, data-driven approach. It thus requires no a priori hypothesis or model of brain activity. Second, by taking account of multiple simultaneous voxel–voxel relationships, ICA detects interacting networks of regions, rather than the single region-dominant (i.e., seed ROI) networks produced by seed-based correlation. Finally, ICA is capable of extracting noise (e.g., scanner, physiological and motion artifacts) from the desired dataset. ICA-based denoising is fully data-driven, automatic (Thomas et al., 2002, Perlbarg et al., 2007, Tohka et al., 2008) and relatively unaffected by different temporal sampling rates (DeLuca et al., 2006), thus avoiding the need for a priori specification of noise that arises in seed-based correlation.
Despite its promise, the unconstrained nature of ICA raises a significant challenge for group-based analyses. If a separate ICA is conducted for each participant in a study, a researcher must first match the components between participants before being able to carry out group analyses (Wang and Peterson, 2008)—a task that is complicated by the unconstrained component order in ICA (Hyvarinen and Oja, 2000). Different numbers of components also can be extracted for different individuals, further complicating the challenge. Accordingly, some have advocated carrying out a single ICA analysis across an entire group of participants (see (Guo and Pagnoni, 2008) and (Calhoun et al., 2009) for reviews on various group ICA methods). This can be accomplished in a relatively efficient manner by carrying out an ICA on a single large dataset by temporally concatenating all individual datasets (Calhoun et al., 2001, Beckmann et al., 2005). This method is referred to as temporal concatenation group ICA (TC-GICA). Based upon group-level components identified by TC-GICA, individual participant components can be generated using approaches such as principal component analysis (PCA) back-projection (Calhoun et al., 2001), or more recently developed GLM dual regression approaches (Filippini et al., 2009, Beckmann et al., 2009). The functionally relevant components of all group-level components are referred to as ICNs.
The potential utility of group-based ICA approaches has been demonstrated by an increasing number of studies examining clinical populations. Clinical samples with Alzheimer's disease or dementia (Greicius et al., 2004, Seeley et al., 2009, Rombouts et al., 2009), schizophrenia (Jafri et al., 2008), depression (Greicius et al., 2007), epilepsy (Zhang et al., 2009), Huntington's disease (Wolf et al., 2008), and amyotrophic lateral sclerosis (Mohammadi et al., 2009) have demonstrated altered ICA-derived ICNs. Despite this promising start, it is important to note that several factors may reduce the reliability of ICA approaches for mapping RSFC. First, the resting state is inherently unconstrained and thus subject to variation related to a participant's state. Second, factors unrelated to the participant can introduce significant variability (e.g., inter-session differences in scanner performance). Third, no ideal means exist for assessing the optimal number of components. Finally, the initial values for ICA estimation are random, which introduces further variability into the findings (Himberg et al., 2004, Yang et al., 2008). In considering these factors, in order to provide biomarkers based on neural circuitry, the test–retest (TRT) reliability of ICA-derived ICNs must first be assessed.
Several groups have addressed the reliability or consistency of ICA. Damoiseaux et al. (2006) demonstrated 10 qualitatively consistent ICNs across subjects by applying tensor ICA to resting state fMRI data from two imaging sessions separated by 5–14 days. They found that (1) spatial patterns of the 10 ICNs from the two sessions were quite consistent by visual inspection, and (2) regions exhibiting the highest RSFC within a component tended to exhibit the least variation across 100 surrogate datasets created via bootstrapping. Recently, using TC-GICA with three initial PCA data reductions, Chen et al. (2008) demonstrated the consistency of ICA-derived ICNs across 5 sessions within 16 days. Two more recent studies also showed high reproducibility and inter-rater selection reliability of an ICA-derived “default mode” network (Meindl et al., 2009, Franco et al., 2009). The above four studies focused only on the spatial consistency of ICNs across occasions within approximately 2 weeks, and they did not quantitatively assess both the voxel-wise intra- and inter-session TRT reliability of each group-level component.
In the current paper, we address the short-term (<45 min) and long-term (> 5 months) TRT of group-level components. We conducted these analyses with a previously published resting state fMRI dataset of 26 participants scanned three times on two different occasions (Shehzad et al., 2009, Zuo et al., 2009). Specifically, we: (a) used the TC-GICA approach to derive group-level components across all participants and sessions, (b) used dual regression to back-reconstruct each group-level component for each of the 3 scans at the individual participant-level, and (c) calculated voxel-wise intra- and inter-session intraclass correlation (ICC) on the basis of these individual-level back-reconstructed components. Based on prior work (Shehzad et al., 2009), we expected to find moderate-to-high TRT reliability for the various ICNs detected except for those components reflecting physiological or scanner-related noise.
While the present work relied upon the group-based TC-GICA approach for detecting components, alternative ICA-based analytic strategies exist. In particular, many studies have employed individual participant based ICA approaches, combined with a template matching for a group analysis (Greicius et al., 2004, Greicius et al., 2007, Seeley et al., 2009, Mohammadi et al., 2009). To date, no study has systematically examined the issue of how easily components identified by group ICA can be detected using individual ICA. Thus, a secondary goal of the present work was to assess the reproducibility of components detected by TC-GICA at the participant-level. Specifically, the reproducibility analysis measured the degree to which components identified using group ICA were present in the results of single-participant ICA. For each participant, we carried out an individual ICA for each of the 3 scans. Then, for each group-level component, a commonly employed template matching procedure (Garrity et al., 2007) was applied to identify the best matching component in each of the three ICA analyses. These best-matched individual-level components were used to evaluate the reproducibility.
Finally, we ranked the group-level components according to their TRT reliability and reproducibility. We hypothesized that functionally relevant ICNs would show high ranks in both TRT reliability and reproducibility. In contrast, we predicted that the components corresponding to various sources of physiological noise and scanner artifact would exhibit both low TRT reliability and reproducibility.
Section snippets
Participants
Twenty-six participants (mean age 20.5 ± 4.8 years, 11 males) were scanned three times. The data were used in our earlier studies to examine the TRT reliability of seed-based correlation (Shehzad et al., 2009) and amplitude measures (Zuo et al., 2009). The participants had no history of psychiatric or neurological illness, as confirmed by clinical assessment. Informed consent was obtained prior to participation. Data were collected according to protocols approved by the institutional review
Results
We carried out 20-component and 42-component TC-GICA. All TRT reliability and reproducibility analyses were performed equally for both ICA decompositions. In this section, we only present the results of the 20-component ICA decomposition. The findings of the 42-component decomposition are summarized in the supplementary materials: Part 1—Reliability and reproducibility for 42-component MELODIC analysis combining dual regression.
Discussion
The present work evaluated the reliability and reproducibility of TC-GICA-derived measures of RSFC, yielding four main results. First, we found moderate-to-high short- and long-term TRT reliability for RSFC measures derived by combining TC-GICA and dual regression. Exceptions to this finding were limited to components corresponding to physiological- and imaging-related artifacts. Second, we found that the degree to which components detected using TC-GICA were reproducible at the level of a
Acknowledgments
This study was supported, in part, by grants from NIMH (R01MH081218), and the Stavros S. Niarchos Foundation to F.X.C., and from the Leon Levy Foundation to M.P.M.; and by gifts from Joseph P. Healey, Linda and Richard Schaps, and Jill and Bob Smith to F.X.C. We desire to express our thanks to Dr. Bharat B. Biswal for helpful advice and Dr. Christian F. Beckmann for invaluable discussions on the dual regression method. Dr. Maarten Mennes helped to improve the readability of the manuscript.
References (88)
- et al.
The resting human brain and motor learning
Curr. Biol.
(2009) - et al.
A review of group ICA for fMRI data and ICA for joint inference of imaging, genetic, and ERP data
Neuroimage
(2009) - et al.
Group independent component analysis reveals consistent resting-state networks across multiple sessions
Brain Res.
(2008) - et al.
Independent component analysis in the presence of noise in fMRI
Magn. Reson. Imaging
(2007) - et al.
fMRI resting state networks define distinct modes of long-distance interactions in the human brain
NeuroImage
(2006) - et al.
Independent component model of the default-mode brain function: assessing the impact of active thinking
Brain Res. Bull.
(2006) - et al.
Intrinsic fluctuations within cortical systems account for intertrial variability in human behavior
Neuron
(2007) How default is the default mode of brain function? further evidence from intrinsic bold signal fluctuations
Neuropsychologia
(Jan 2006)- et al.
Large-amplitude, spatially correlated fluctuations in bold fMRI signals during extended rest and early sleep stages
Magn. Reson. Imaging
(2006) - et al.
Resting-state functional connectivity in major depression: abnormally increased contributions from subgenual cingulate cortex and thalamus
Biol. Psychiatry
(2007)
A unified framework for group independent component analysis for multi-subject fMRI data
NeuroImage
Validating the independent components of neuroimaging time series via clustering and visualization
NeuroImage
Independent component analysis: algorithms and applications
Neural Networks
A method for functional network connectivity among spatially independent resting-state components in schizophrenia
NeuroImage
Improved optimization for the robust and accurate linear registration and motion correction of brain images
NeuroImage
Competition between functional brain networks mediates behavioral variability
NeuroImage
Independent component analysis of nondeterministic fMRI signal sources
NeuroImage
Default mode network as revealed with multiple methods for resting-state functional MRI analysis
J. Neurosci. Methods
Mapping the functional connectivity of anterior cingulate cortex
NeuroImage
The relative involvement of anterior cingulate and prefrontal cortex in attentional control depends on nature of conflict
Brain Res. Cogn. Brain Res.
Changes of resting state brain networks in amyotrophic lateral sclerosis
Exp. Neurol.
The impact of global signal regression on resting state correlations: are anti-correlated networks introduced?
NeuroImage
CORSICA: correction of structured noise in fMRI by automatic identification of ICA components
Magn. Reson. Imaging
Modeling by shortest data description
Automatica
Functional connectivity of the human amygdala using resting state fMRI
NeuroImage
Neurodegenerative diseases target large-scale human brain networks
Neuron
Modeling brain activation patterns for the default and cognitive states
NeuroImage
Noise reduction in bold-based fMRI using component analysis
NeuroImage
Automatic independent component labeling for artifact removal in fMRI
NeuroImage
Correlations and anticorrelations in resting-state functional connectivity MRI: a quantitative comparison of preprocessing strategies
NeuroImage
Aberrant connectivity of lateral prefrontal networks in presymptomatic Huntington's disease
Exp. Neurol.
Impaired attention network in temporal lobe epilepsy: a resting fMRI study
Neurosci. Lett.
Information theory and an extension of the maximum likelihood principle
Second International Symposium on Information Theory
Probabilistic independent component analysis for functional magnetic resonance imaging
IEEE Trans. Med. Imag.
Investigations into resting-state connectivity using independent component analysis
Philos. Trans. R Soc. Lond, B, Biol. Sci.
Functional connectivity in the motor cortex of resting human brain using echo-planar MRI
Magn. Reson. Med.
Intrinsic brain activity in altered states of consciousness: how conscious is the default mode of brain function
Ann. N. Y. Acad. Sci.
Default-mode brain dysfunction in mental disorders: a systematic review
Neurosci. Biobehav. Rev.
The brain's default network: anatomy, function, and relevance to disease
Ann. N. Y. Acad. Sci.
Cortical hubs revealed by intrinsic functional connectivity: mapping, assessment of stability, and relation to Alzheimer's disease
J. Neurosci.
A method for making group inferences from functional MRI data using independent component analysis
Hum. Brain Mapp.
Alcohol intoxication effects on simulated driving: exploring alcohol-dose effects on brain activation using functional MRI
Neuropsychopharmacology
Cited by (641)
Prenatal Exposure to Maternal Mood Entropy Is Associated With a Weakened and Inflexible Salience Network in Adolescence
2024, Biological Psychiatry: Cognitive Neuroscience and Neuroimaging