To date, there has been little investigation of the neurobiological basis of emotion processing abnormalities in psychiatric populations. We have previously discussed two neural systems: 1) a ventral system, including the amygdala, insula, ventral striatum, ventral anterior cingulate gyrus, and prefrontal cortex, for identification of the emotional significance of a stimulus, production of affective states, and automatic regulation of emotional responses; and 2) a dorsal system, including the hippocampus, dorsal anterior cingulate gyrus, and prefrontal cortex, for the effortful regulation of affective states and subsequent behavior. In this critical review, we have examined evidence from studies employing a variety of techniques for distinct patterns of structural and functional abnormalities in these neural systems in schizophrenia, bipolar disorder, and major depressive disorder. In each psychiatric disorder, the pattern of abnormalities may be associated with specific symptoms, including emotional flattening, anhedonia, and persecutory delusions in schizophrenia, prominent mood swings, emotional lability, and distractibility in bipolar disorder during depression and mania, and with depressed mood and anhedonia in major depressive disorder. We suggest that distinct patterns of structural and functional abnormalities in neural systems important for emotion processing are associated with specific symptoms of schizophrenia and bipolar and major depressive disorder.