The perception of facial affect engages a distributed cortical network. We used functional magnetic resonance imaging and dynamic causal modeling to characterize effective connectivity during explicit (conscious) categorization of affective stimuli in the human brain. Specifically, we examined the modulation of connectivity from posterior regions of the face-processing network to the lateral ventral prefrontal cortex (VPFC) during affective categorization and we tested for a potential role of the amygdala (AMG) in mediating this modulation. We found that explicit processing of facial affect led to prominent modulation (increase) in the effective connectivity from the inferior occipital gyrus (IOG) to the VPFC, while there was less evidence for modulation of the afferent connections from fusiform gyrus and AMG to VPFC. More specifically, the forward connection from IOG to the VPFC exhibited a selective increase under anger (as opposed to fear or sadness). Furthermore, Bayesian model comparison suggested that the modulation of afferent connections to the VPFC was mediated directly by facial affect, as opposed to an indirect modulation mediated by the AMG. Our results thus suggest that affective information is conveyed to the VPFC along multiple parallel pathways and that AMG activity is not sufficient to account for the gating of information transfer to the VPFC during explicit emotional processing.