In this study, we explored to what extent brain abnormalities can be identified in specific brain structures of patients suffering from late onset depression. We examined the structural difference in regional gray and white matter volume between 14 community-dwelling patients suffering from geriatric depression and 20 age-matched non-depressed normal subjects by voxel-based morphometry (VBM) based on magnetic resonance imaging. All subjects also underwent an extensive neuropsychological assessment. Compared with control subjects, patients with depression were impaired in measures of verbal and visual memory, construction, executive ability, and information-processing speed. VBM of gray matter revealed a significant decrease of volume in the right rostral hippocampus, in the right amygdala and in the medial orbito-frontal cortex (gyrus rectus) bilaterally. In the correlation analysis of gray matter volume with the score of the geriatric depression scale, we observed a negative correlation with the medial orbito-frontal cortex (gyrus rectus) bilaterally. There were no differences in white matter volumes between patients with depression and healthy control subjects. The most important limitation of this study was sample size. A larger sample size may have improved detection of changes not reaching significance. Furthermore, our results may not be generalizable across depression severity or to hospitalized patients. The findings are consistent with our hypothesis that depression in the elderly is associated with local gray matter dysfunction.