Increasing evidence supports the existence of distinct neural systems that subserve two dimensions of affect--arousal and valence. Ten adult participants underwent functional magnetic resonance imaging during which they were presented a range of standardized faces and then asked, during the scan, to rate the emotional expressions of the faces along the dimensions of arousal and valence. Lower ratings of arousal accompanied greater activity in the amygdala complex, cerebellum, dorsal pons, and right medial prefrontal cortex (mPFC). More negative ratings of valence accompanied greater activity in the dorsal anterior cingulate (dACC) and parietal cortices. Extreme ratings of valence (highly positive and highly negative ratings) accompanied greater activity in the temporal cortex and fusiform gyrus. Building on an empirical literature which suggests that the amygdala serves as a salience and ambiguity detector, we interpret our findings as showing that a face rated as having low arousal is more ambiguous and a face rated as having extreme valence is more personally salient. This explains how both low arousal and extreme valence lead to greater activation of an ambiguity/salience system subserved by the amygdala, cerebellum, and dorsal pons. In addition, the right medial prefrontal cortex appears to down-regulate individual ratings of arousal, whereas the fusiform and related temporal cortices seem to up-regulate individual assessments of extreme valence when individual ratings are studied relative to group reference ratings for each stimulus. The simultaneous assessment of the effects of arousal and valence proved essential for the identification of neural systems contributing to the processing of emotional faces.