Investigating the predictive value of whole-brain structural MR scans in autism: A pattern classification approach

Abstract

Autistic spectrum disorder (ASD) is accompanied by subtle and spatially distributed differences in brain anatomy that are difficult to detect using conventional mass-univariate methods (e.g., VBM). These require correction for multiple comparisons and hence need relatively large samples to attain sufficient statistical power. Reports of neuroanatomical differences from relatively small studies are thus highly variable. Also, VBM does not provide predictive value, limiting its diagnostic value.

Here, we examined neuroanatomical networks implicated in ASD using a whole-brain classification approach employing a support vector machine (SVM) and investigated the predictive value of structural MRI scans in adults with ASD. Subsequently, results were compared between SVM and VBM. We included 44 male adults; 22 diagnosed with ASD using “gold-standard” research interviews and 22 healthy matched controls.

SVM identified spatially distributed networks discriminating between ASD and controls. These included the limbic, frontal-striatal, fronto-temporal, fronto-parietal and cerebellar systems. SVM applied to gray matter scans correctly classified ASD individuals at a specificity of 86.0% and a sensitivity of 88.0%. Cases (68.0%) were correctly classified using white matter anatomy. The distance from the separating hyperplane (i.e., the test margin) was significantly related to current symptom severity. In contrast, VBM revealed few significant between-group differences at conventional levels of statistical stringency.

We therefore suggest that SVM can detect subtle and spatially distributed differences in brain networks between adults with ASD and controls. Also, these differences provide significant predictive power for group membership, which is related to symptom severity.

Introduction

Autistic spectrum disorder (ASD) is a highly genetic neurodevelopmental condition (Bailey et al., 1996, Bolton and Rutter, 1990, Folstein and Rutter, 1977), which is characterized by a triad of symptoms. These are impaired social communication, social reciprocity and repetitive and stereotypic behavior (Gillberg, 1993, Wing, 1997). The behavioral phenotype of ASD is well described but less is known about its etiology and pathogenesis, although neuroimaging studies have repeatedly highlighted the potential role of a number of brain regions and neural systems (for a review, see Amaral et al., 2008, Toal et al., 2005).

Differences in gray matter (GM) volume have been reported by several previous studies using “regions of interest” (ROI) approaches involving manually tracing brain regions as well as automated voxel-based morphometry (VBM) of the entire brain (Ashburner and Friston, 2000). The findings of previous studies, while reporting variable results, predominantly point toward abnormalities in frontal, parietal and limbic regions as well as the basal ganglia and the cerebellum (McAlonan et al., 2005, McAlonan et al., 2002, Rojas et al., 2006, Waiter et al., 2004). Some have also reported that anatomical differences in some of these regions are correlated with clinical severity (Rojas et al., 2006, Schmitz et al., 2006). There is also evidence for differences in white matter volume (Herbert et al., 2004, McAlonan et al., 2002) and microstructural integrity, which may affect interhemispheric “connectivity” (Hardan et al., 2000, Waiter et al., 2004), inhibition of fronto-striatal systems (McAlonan et al., 2002) and the cerebellar circuitry (Catani et al., 2008). Hence, there is increasing evidence that people with ASD have differences in brain morphology. Nevertheless, as noted above, the results of many studies are in disagreement. This may be explained simply on the basis of factors such as clinical heterogeneity between studies. However, it may also indicate that differences in brain anatomy are relatively subtle and widespread; if so, this presents significant challenges to whichever analytic technique is employed.

Region of interest (ROI, i.e., manual tracing) approaches provide higher statistical power than voxel-wise analysis methods such as VBM because only relatively few brain regions are investigated and so less correction for multiple comparisons is required. On the other hand, they have relatively low exploratory power because they restrict their focus to a small number of specific regions. In contrast, whole-brain, mass-univariate analyses (e.g., VBM) offer high exploratory power but with only moderate statistical power, as corrections for multiple comparisons are required in order to limit the occurrence of false positives. Thus, mass-univariate approaches may be too conservative to detect subtle morphological differences in the brain (especially with relatively small sample sizes). In addition, VBM-type approaches do not offer predictive value, which may be of diagnostic relevance.

Currently, ASD is diagnosed solely on the basis of behavioral criteria. The behavioral diagnosis of ASD is however often time consuming and can be problematic, particularly in adults. For example, the “gold-standard” diagnostic instruments for ASD such as the Autism Diagnostic Observation Schedule (ADOS; Lord, 1989) and/or the Autism Diagnostic Interview (ADI-R; Lord et al., 1994) are dependent on the skill of the examiner in, respectively, observing current symptoms and eliciting a developmental history from informants who know the person very well. However, in adults, current symptoms are often modulated by coping strategies developed over the life span, and retrospective accounts of past symptoms rely on an informant being both reliable and available. Also, different biological (e.g., genetic/environmental) etiologies might result in the same behavioral phenotype (sometimes referred to as the “autisms”; Geschwind and Levitt, 2007), but this is undetectable using behavioral measures alone. Hence, some have suggested that endophenotypes need to be identified to aid genetic studies. To enable this, new approaches are required (perhaps especially in adults) to combine biological information with behavioral measures.

We therefore investigated the predictive value of neuroanatomical gray and white matter MRI scans in ASD using support vector machines (SVMs). SVMs belong to the general machine learning-based pattern recognition techniques that can be used to classify data by differentiating between two or more classes (e.g., patients vs. controls). SVMs are initially “trained” on the basis of a well-characterized sample or a subset of the data in order to establish a mathematical criterion that best distinguishes the groups (i.e., training phase). Once a so-called “decision function” or “hyperplane” is learned from the training data, it can be used to predict the class of a new test example (e.g., whether a new participant has ASD). Importantly for studies such as the current one, SVM also provides numerical indicators for specificity (i.e., the proportion of people without disease displaying a negative test result) and sensitivity (i.e., proportion of people with disease displaying a positive result), thus offering predictive value. In addition, and in contrast to VBM-based approaches, SVM is multivariate and so takes into account inter-regional correlations. This may be of particular relevance to ASD, as individuals with the disorder most likely have abnormalities in the development of neural systems – rather than isolated regions – leading to differences in intra-regional correlations of brain metabolism, function and anatomy (Horwitz et al., 1988, Koshino et al., 2005, McAlonan et al., 2002). SVM may therefore be particularly suited to detect subtle differences in the morphology of brain systems.

A prior study using discriminate function analysis reported that some pre-selected differences in brain anatomy (e.g., volume of cerebellum and whole-brain gray and white matter) correctly classified 95% of very young people with ASD (Akshmoomoff et al., 2004). To date, however, few studies have employed SVMs in the analysis of structural MRI data in humans, and these have mainly been used to classify patients with Alzheimer's disease (Kloppel et al., 2008, Vemuri et al., 2008) and mild cognitive impairment (Davatzikos et al., 2008, Teipel et al., 2007). Other studies have also reported that differences in white matter shape, the size of corpus callosum (Akshoomoff et al., 2004) and cortical thickness (Singh et al., 2008) distinguish young people with autism from controls. These prior studies employing discriminate function and/or SVM were important first steps. However, they were all based on a priori selected features rather than whole-brain data, and so this significantly impacted on their exploratory power and ability to investigate the role of numerous other brain regions and systems implicated in ASD. In addition, none of these studies validated their classification of individuals with ASD by relating classification to symptom severity.

Thus, we used an SVM classification approach to discriminate adults with ASD from controls based on whole-brain gray and white matter anatomy. Also, we related classification results to symptom severity. Lastly, we compared the results obtained by our SVM analysis with that from conventional VBM mass-univariate analysis.