Elsevier

NeuroImage

Volume 49, Issue 3, 1 February 2010, Pages 2163-2177
NeuroImage

Reliable intrinsic connectivity networks: Test–retest evaluation using ICA and dual regression approach

https://doi.org/10.1016/j.neuroimage.2009.10.080Get rights and content

Abstract

Functional connectivity analyses of resting-state fMRI data are rapidly emerging as highly efficient and powerful tools for in vivo mapping of functional networks in the brain, referred to as intrinsic connectivity networks (ICNs). Despite a burgeoning literature, researchers continue to struggle with the challenge of defining computationally efficient and reliable approaches for identifying and characterizing ICNs. Independent component analysis (ICA) has emerged as a powerful tool for exploring ICNs in both healthy and clinical populations. In particular, temporal concatenation group ICA (TC-GICA) coupled with a back-reconstruction step produces participant-level resting state functional connectivity maps for each group-level component. The present work systematically evaluated the test–retest reliability of TC-GICA derived RSFC measures over the short-term (< 45 min) and long-term (5–16 months). Additionally, to investigate the degree to which the components revealed by TC-GICA are detectable via single-session ICA, we investigated the reproducibility of TC-GICA findings. First, we found moderate-to-high short- and long-term test–retest reliability for ICNs derived by combining TC-GICA and dual regression. Exceptions to this finding were limited to physiological- and imaging-related artifacts. Second, our reproducibility analyses revealed notable limitations for template matching procedures to accurately detect TC-GICA based components at the individual scan level. Third, we found that TC-GICA component's reliability and reproducibility ranks are highly consistent. In summary, TC-GICA combined with dual regression is an effective and reliable approach to exploratory analyses of resting state fMRI data.

Introduction

Functional connectivity of resting-state fMRI data is rapidly emerging as a highly efficient and powerful tool for in vivo mapping of neural circuitry in the human brain. Essentially, resting state functional connectivity (RSFC) approaches detect coherent patterns of low-frequency (< 0.1 Hz) fluctuations in the resting state BOLD signal, referred to as “intrinsic connectivity networks” (ICNs). Despite a burgeoning literature demonstrating the utility of RSFC approaches (Fox and Raichle, 2007), researchers continue to struggle with defining computationally efficient and reliable approaches for identifying and characterizing ICNs.

Inspired by a seminal demonstration of RSFC within the motor system (Biswal et al., 1995), seed-based correlation represents the dominant approach for studying RSFC. This method detects ICNs by identifying voxels whose timeseries significantly correlate with the mean timeseries of voxels within an a priori seed region of interest (ROI). In addition to generating highly detailed maps of complex functional systems (Margulies et al., 2007, Zhang et al., 2008, Di Martino et al., 2008, Roy et al., 2009, Nioche et al., 2009), seed-based correlation analyses are commonly used to identify abnormalities in ICNs related to psychopathology (Greicius, 2008, Broyd et al., 2009), as well as relationships between RSFC measures and individual differences in behavioral (Kelly et al., 2008, Fox et al., 2007) and trait characteristics (Di Martino et al., 2009). Overall, this mass-univariate approach has proven to be a powerful, efficient, and reliable tool for neuroimaging (Shehzad et al., 2009).

Seed-based analytic approaches are not without limitation, however. Most notably, the ICNs derived with seed-based correlation are highly dependent upon choice of a seed ROI. Further, the seed ROI timeseries includes indeterminate noise, in contrast to the idealized task design matrix used in task-based analyses. While several preprocessing strategies commonly are employed to remove noise (e.g., regressing out the global signal), such corrections can affect the data (e.g., inducing artifactual negative correlations) (Murphy et al., 2009). Finally, as a general linear model (GLM)-based univariate method, seed-based correlation disregards relationships among multiple voxels (i.e., it only analyzes the relationship between the seed timeseries and one other voxel at a time).

In light of these issues, neuroimaging researchers have adopted a multivariate signal processing method known as independent component analysis (ICA) to explore the spatial-temporal properties of resting state fMRI data (Kiviniemi, 2003, van de Ven et al., 2004, Beckmann et al., 2005). In theory, without any explicit a priori knowledge, ICA aims to separate spatially (spatial ICA: sICA) or temporally (temporal ICA: tICA) independent patterns from their linearly mixed BOLD signals via maximization of mutual independence among components (Stone et al., 1999). Generally, sICA is the more appropriate choice for analyzing resting state fMRI data given the small number of time points included in most fMRI datasets. Thus, we also used sICA in this paper. For simplicity, in the remainder of this paper, we refer to sICA as ICA.

ICA offers several potential advantages over seed-based correlation. First, as noted above, ICA is a multivariate, data-driven approach. It thus requires no a priori hypothesis or model of brain activity. Second, by taking account of multiple simultaneous voxel–voxel relationships, ICA detects interacting networks of regions, rather than the single region-dominant (i.e., seed ROI) networks produced by seed-based correlation. Finally, ICA is capable of extracting noise (e.g., scanner, physiological and motion artifacts) from the desired dataset. ICA-based denoising is fully data-driven, automatic (Thomas et al., 2002, Perlbarg et al., 2007, Tohka et al., 2008) and relatively unaffected by different temporal sampling rates (DeLuca et al., 2006), thus avoiding the need for a priori specification of noise that arises in seed-based correlation.

Despite its promise, the unconstrained nature of ICA raises a significant challenge for group-based analyses. If a separate ICA is conducted for each participant in a study, a researcher must first match the components between participants before being able to carry out group analyses (Wang and Peterson, 2008)—a task that is complicated by the unconstrained component order in ICA (Hyvarinen and Oja, 2000). Different numbers of components also can be extracted for different individuals, further complicating the challenge. Accordingly, some have advocated carrying out a single ICA analysis across an entire group of participants (see (Guo and Pagnoni, 2008) and (Calhoun et al., 2009) for reviews on various group ICA methods). This can be accomplished in a relatively efficient manner by carrying out an ICA on a single large dataset by temporally concatenating all individual datasets (Calhoun et al., 2001, Beckmann et al., 2005). This method is referred to as temporal concatenation group ICA (TC-GICA). Based upon group-level components identified by TC-GICA, individual participant components can be generated using approaches such as principal component analysis (PCA) back-projection (Calhoun et al., 2001), or more recently developed GLM dual regression approaches (Filippini et al., 2009, Beckmann et al., 2009). The functionally relevant components of all group-level components are referred to as ICNs.

The potential utility of group-based ICA approaches has been demonstrated by an increasing number of studies examining clinical populations. Clinical samples with Alzheimer's disease or dementia (Greicius et al., 2004, Seeley et al., 2009, Rombouts et al., 2009), schizophrenia (Jafri et al., 2008), depression (Greicius et al., 2007), epilepsy (Zhang et al., 2009), Huntington's disease (Wolf et al., 2008), and amyotrophic lateral sclerosis (Mohammadi et al., 2009) have demonstrated altered ICA-derived ICNs. Despite this promising start, it is important to note that several factors may reduce the reliability of ICA approaches for mapping RSFC. First, the resting state is inherently unconstrained and thus subject to variation related to a participant's state. Second, factors unrelated to the participant can introduce significant variability (e.g., inter-session differences in scanner performance). Third, no ideal means exist for assessing the optimal number of components. Finally, the initial values for ICA estimation are random, which introduces further variability into the findings (Himberg et al., 2004, Yang et al., 2008). In considering these factors, in order to provide biomarkers based on neural circuitry, the test–retest (TRT) reliability of ICA-derived ICNs must first be assessed.

Several groups have addressed the reliability or consistency of ICA. Damoiseaux et al. (2006) demonstrated 10 qualitatively consistent ICNs across subjects by applying tensor ICA to resting state fMRI data from two imaging sessions separated by 5–14 days. They found that (1) spatial patterns of the 10 ICNs from the two sessions were quite consistent by visual inspection, and (2) regions exhibiting the highest RSFC within a component tended to exhibit the least variation across 100 surrogate datasets created via bootstrapping. Recently, using TC-GICA with three initial PCA data reductions, Chen et al. (2008) demonstrated the consistency of ICA-derived ICNs across 5 sessions within 16 days. Two more recent studies also showed high reproducibility and inter-rater selection reliability of an ICA-derived “default mode” network (Meindl et al., 2009, Franco et al., 2009). The above four studies focused only on the spatial consistency of ICNs across occasions within approximately 2 weeks, and they did not quantitatively assess both the voxel-wise intra- and inter-session TRT reliability of each group-level component.

In the current paper, we address the short-term (<45 min) and long-term (> 5 months) TRT of group-level components. We conducted these analyses with a previously published resting state fMRI dataset of 26 participants scanned three times on two different occasions (Shehzad et al., 2009, Zuo et al., 2009). Specifically, we: (a) used the TC-GICA approach to derive group-level components across all participants and sessions, (b) used dual regression to back-reconstruct each group-level component for each of the 3 scans at the individual participant-level, and (c) calculated voxel-wise intra- and inter-session intraclass correlation (ICC) on the basis of these individual-level back-reconstructed components. Based on prior work (Shehzad et al., 2009), we expected to find moderate-to-high TRT reliability for the various ICNs detected except for those components reflecting physiological or scanner-related noise.

While the present work relied upon the group-based TC-GICA approach for detecting components, alternative ICA-based analytic strategies exist. In particular, many studies have employed individual participant based ICA approaches, combined with a template matching for a group analysis (Greicius et al., 2004, Greicius et al., 2007, Seeley et al., 2009, Mohammadi et al., 2009). To date, no study has systematically examined the issue of how easily components identified by group ICA can be detected using individual ICA. Thus, a secondary goal of the present work was to assess the reproducibility of components detected by TC-GICA at the participant-level. Specifically, the reproducibility analysis measured the degree to which components identified using group ICA were present in the results of single-participant ICA. For each participant, we carried out an individual ICA for each of the 3 scans. Then, for each group-level component, a commonly employed template matching procedure (Garrity et al., 2007) was applied to identify the best matching component in each of the three ICA analyses. These best-matched individual-level components were used to evaluate the reproducibility.

Finally, we ranked the group-level components according to their TRT reliability and reproducibility. We hypothesized that functionally relevant ICNs would show high ranks in both TRT reliability and reproducibility. In contrast, we predicted that the components corresponding to various sources of physiological noise and scanner artifact would exhibit both low TRT reliability and reproducibility.

Section snippets

Participants

Twenty-six participants (mean age 20.5 ± 4.8 years, 11 males) were scanned three times. The data were used in our earlier studies to examine the TRT reliability of seed-based correlation (Shehzad et al., 2009) and amplitude measures (Zuo et al., 2009). The participants had no history of psychiatric or neurological illness, as confirmed by clinical assessment. Informed consent was obtained prior to participation. Data were collected according to protocols approved by the institutional review

Results

We carried out 20-component and 42-component TC-GICA. All TRT reliability and reproducibility analyses were performed equally for both ICA decompositions. In this section, we only present the results of the 20-component ICA decomposition. The findings of the 42-component decomposition are summarized in the supplementary materials: Part 1—Reliability and reproducibility for 42-component MELODIC analysis combining dual regression.

Discussion

The present work evaluated the reliability and reproducibility of TC-GICA-derived measures of RSFC, yielding four main results. First, we found moderate-to-high short- and long-term TRT reliability for RSFC measures derived by combining TC-GICA and dual regression. Exceptions to this finding were limited to components corresponding to physiological- and imaging-related artifacts. Second, we found that the degree to which components detected using TC-GICA were reproducible at the level of a

Acknowledgments

This study was supported, in part, by grants from NIMH (R01MH081218), and the Stavros S. Niarchos Foundation to F.X.C., and from the Leon Levy Foundation to M.P.M.; and by gifts from Joseph P. Healey, Linda and Richard Schaps, and Jill and Bob Smith to F.X.C. We desire to express our thanks to Dr. Bharat B. Biswal for helpful advice and Dr. Christian F. Beckmann for invaluable discussions on the dual regression method. Dr. Maarten Mennes helped to improve the readability of the manuscript.

References (88)

  • GuoY. et al.

    A unified framework for group independent component analysis for multi-subject fMRI data

    NeuroImage

    (2008)
  • HimbergJ. et al.

    Validating the independent components of neuroimaging time series via clustering and visualization

    NeuroImage

    (2004)
  • HyvarinenA. et al.

    Independent component analysis: algorithms and applications

    Neural Networks

    (2000)
  • JafriM.J. et al.

    A method for functional network connectivity among spatially independent resting-state components in schizophrenia

    NeuroImage

    (2008)
  • JenkinsonM. et al.

    Improved optimization for the robust and accurate linear registration and motion correction of brain images

    NeuroImage

    (2002)
  • KellyA.M.C. et al.

    Competition between functional brain networks mediates behavioral variability

    NeuroImage

    (2008)
  • KiviniemiV.

    Independent component analysis of nondeterministic fMRI signal sources

    NeuroImage

    (2003)
  • LongX.Y. et al.

    Default mode network as revealed with multiple methods for resting-state functional MRI analysis

    J. Neurosci. Methods

    (2008)
  • MarguliesD.S. et al.

    Mapping the functional connectivity of anterior cingulate cortex

    NeuroImage

    (2007)
  • MilhamM.P. et al.

    The relative involvement of anterior cingulate and prefrontal cortex in attentional control depends on nature of conflict

    Brain Res. Cogn. Brain Res.

    (2001)
  • MohammadiB. et al.

    Changes of resting state brain networks in amyotrophic lateral sclerosis

    Exp. Neurol.

    (2009)
  • MurphyK. et al.

    The impact of global signal regression on resting state correlations: are anti-correlated networks introduced?

    NeuroImage

    (2009)
  • PerlbargV. et al.

    CORSICA: correction of structured noise in fMRI by automatic identification of ICA components

    Magn. Reson. Imaging

    (2007)
  • RissanenJ.

    Modeling by shortest data description

    Automatica

    (1978)
  • RoyA.K. et al.

    Functional connectivity of the human amygdala using resting state fMRI

    NeuroImage

    (2009)
  • SeeleyW.W. et al.

    Neurodegenerative diseases target large-scale human brain networks

    Neuron

    (2009)
  • Steyn-RossM.L. et al.

    Modeling brain activation patterns for the default and cognitive states

    NeuroImage

    (2009)
  • ThomasC.G. et al.

    Noise reduction in bold-based fMRI using component analysis

    NeuroImage

    (2002)
  • TohkaJ. et al.

    Automatic independent component labeling for artifact removal in fMRI

    NeuroImage

    (2008)
  • WeissenbacherA. et al.

    Correlations and anticorrelations in resting-state functional connectivity MRI: a quantitative comparison of preprocessing strategies

    NeuroImage

    (2009)
  • WolfR.C. et al.

    Aberrant connectivity of lateral prefrontal networks in presymptomatic Huntington's disease

    Exp. Neurol.

    (2008)
  • ZhangZ. et al.

    Impaired attention network in temporal lobe epilepsy: a resting fMRI study

    Neurosci. Lett.

    (2009)
  • AkaikeH.

    Information theory and an extension of the maximum likelihood principle

    Second International Symposium on Information Theory

    (1973)
  • Andersson, J.L.R., Jenkinson, M., Smith, S., 2007. Non-linear registration, aka spatial normalisation. Tech. Rep....
  • BeckmannC.F. et al.

    Probabilistic independent component analysis for functional magnetic resonance imaging

    IEEE Trans. Med. Imag.

    (2004)
  • BeckmannC.F. et al.

    Investigations into resting-state connectivity using independent component analysis

    Philos. Trans. R Soc. Lond, B, Biol. Sci.

    (2005)
  • Beckmann, C., Mackay, C., Filippini, N., Smith, S., 2009. Group comparison of resting-state fMRI data using...
  • BiswalB. et al.

    Functional connectivity in the motor cortex of resting human brain using echo-planar MRI

    Magn. Reson. Med.

    (1995)
  • BolyM. et al.

    Intrinsic brain activity in altered states of consciousness: how conscious is the default mode of brain function

    Ann. N. Y. Acad. Sci.

    (2008)
  • BroydS. et al.

    Default-mode brain dysfunction in mental disorders: a systematic review

    Neurosci. Biobehav. Rev.

    (2009)
  • BucknerR.L. et al.

    The brain's default network: anatomy, function, and relevance to disease

    Ann. N. Y. Acad. Sci.

    (2008)
  • BucknerR.L. et al.

    Cortical hubs revealed by intrinsic functional connectivity: mapping, assessment of stability, and relation to Alzheimer's disease

    J. Neurosci.

    (2009)
  • CalhounV.D. et al.

    A method for making group inferences from functional MRI data using independent component analysis

    Hum. Brain Mapp.

    (2001)
  • CalhounV.D. et al.

    Alcohol intoxication effects on simulated driving: exploring alcohol-dose effects on brain activation using functional MRI

    Neuropsychopharmacology

    (2004)
  • Cited by (641)

    View all citing articles on Scopus
    View full text