The present event-related fMRI study examined in ten healthy participants neural correlates of recognition memory for intact, rearranged, and new pairs of visual stimuli. The correct categorization of both intact and rearranged pairs relative to pairs of new, never presented before stimuli relies on the retrieval of associative information whereas the discrimination of rearranged pairs from intact pairs additionally involves a recall-to-reject process by which subjects retrieve additional information to reach a recognition decision. Relative to new pairs, both intact and rearranged pairs yielded activation in inferior prefrontal cortex bilaterally and left dorsolateral prefrontal cortex. Relative to intact pairs, rearranged pairs were associated with increased activity in left dorsolateral prefrontal cortex. The latter could reflect the neural correlates of a recall-to-reject process, a pivotal process of associative recognition memory.