The putative role of the so-called Visual Word Form Area (VWFA) during reading remains under debate. For some authors, this region is specifically involved in a pre-lexical processing of words and pseudowords, whereas such specificity is challenged by others given the VWFA involvement during both non-word reading and word listening. Here, we further investigated this issue, measuring BOLD variations and their lateralization with fMRI during word and non-word reading, in order to evaluate the lexicality effect, and during reading and listening of words, in order to evaluate the impact of stimulus delivery modality on word processing networks. Region of interest (ROI) analysis was first performed in three target areas: 1-VWFA as defined by a meta-analysis of the word reading literature, 2-a middle temporal area (T2) found co-activated by both word reading and listening, 3-an inferior occipital area (OI) belonging to the unimodal visual cortex of the inferior occipital gyrus. VWFA activity was found not different between word and non-word reading but was more leftward lateralized during word reading due to a reduction of activity in the VWFA right counterpart. A similar larger leftward lateralization during word reading was also uncovered in the T2 ROI but was related to a larger left side activity. Such a lexicality effect was not observed in the OI ROI. By contrast, BOLD increases during listening were restricted to the left VWFA and T2 ROIs. Voxel-based analysis (SPM99) showed that semantic areas were more active during word than non-word reading and co-activated by both reading and listening, exhibiting a left lateralized activity in all tasks. These results indicate that the left VWFA would be the place where visual and verbal representations bind under the control of left semantic areas.