Prolonged exposure to complex stimuli, such as faces, biases perceptual decisions toward nonadapted, dissimilar stimuli, leading to contrastive aftereffects. Here we tested the neural correlates of this perceptual bias using a functional magnetic resonance imaging adaptation (fMRIa) paradigm. Adaptation to a face or hand stimulus led to aftereffects by biasing the categorization of subsequent ambiguous face/hand composite stimuli away from the adaptor category. The simultaneously observed fMRIa in the face-sensitive fusiform face area (FFA) and in the body-part-sensitive extrastriate body area (EBA) depended on the behavioral response of the subjects: adaptation to the preferred stimulus of the given area led to larger signal reduction during trials when it biased perception than during trials when it was less effective. Activity in two frontal areas correlated positively with the activity patterns in FFA and EBA. Based on our novel adaptation paradigm, the results suggest that the adaptation-induced aftereffects are mediated by the relative activity of category-sensitive areas of the human brain as demonstrated by fMRI.