Borderline personality disorder (BPD) is characterized by interpersonal disturbances, but the neurocognitive aspects of these symptoms are poorly understood. We hypothesized that patients with BPD have impaired perception of emotional expressions, which are related to symptoms of interpersonal dysfunction. To control potential confounding factors, this study excluded subjects with comorbid diagnoses known to be associated with impaired affect perception. We tested 43 outpatients with BPD and 26 healthy controls on emotion recognition tasks (facial, prosodic, and integrated facial/prosodic), nonemotional facial feature recognition, and interpersonal antagonism (Buss-Durkee Hostility Index). Patients with BPD showed normal ability to recognize isolated facial or prosodic emotions but had impaired recognition of emotions in integrated facial/prosodic stimuli, as well as impaired discrimination of nonemotional facial features. In patients with BPD, impaired recognition of integrated emotional stimuli was associated with interpersonal antagonism, particularly suspiciousness and assaultiveness. These results suggest that patients with BPD have deficits in higher order integration of social information, which may be related to some of the more serious symptoms of the disorder.