Although the basal ganglia are thought to be important in recognizing emotion, there is contradictory evidence as to whether patients with Parkinson's disease (PD) have deficits in recognizing facial expressions. In addition, few studies have examined their ability to recognize emotion from non-visual stimuli, such as voices. We examined the ability of PD patients and age-matched controls to recognize emotion in three different modalities: facial, prosodic, and written verbal stimuli. Compared to controls, PD patients showed deficits in recognizing fear and disgust in facial expressions. These impairments were not seen in their recognition of prosodic or written verbal stimuli. This modality-specific deficit suggests that the neural substrates for recognizing emotion from different modalities are not fully identical.