We hypothesized that brain activation during encoding and retrieval of visual material differed between epilepsy patients with hippocampal sclerosis (HS) and healthy controls. Eleven patients with epilepsy and HS and nine age- and education-matched control subjects were tested during functional MRI recording. A three-block design for visuospatial memory encoding and retrieval and an interference interval longer than 1 minute without memory tasks were used. All subjects revealed parietal, occipital, and prefrontal activation patterns during encoding. Interference revealed parietal more than occipital activation, whereas retrieval revealed asymmetrical frontal and parietal activation. Patients demonstrated a relative increase in occipitoparietal versus frontal cortical activation as compared with controls. Memory performance did not differ between patients and controls. The increased activation in occipitoparietal versus frontal areas in the patients suggests cortical reorganization of visuospatial recognition memory in epilepsy patients with HS. The study is limited by other factors that may contribute to the results, for example, antiepileptic drugs, effects of greater cognitive effort allocated in patients than controls, and possibly subclinical epileptic activity. However, normal visuospatial memory performance in our patients with HS suggests successful network plasticity.