Humans consistently land their first saccade to a face at a preferred fixation location (PFL). Humans also typically process faces as wholes, as evidenced by perceptual effects such as the composite face effect (CFE). However, not known is whether an individual's tendency to process faces as wholes varies with their gaze patterns on the face. Here, we investigated variation of the CFE with the PFL. We compared the strength of the CFE for two groups of observers who were screened to have their PFLs either higher up, closer to the eyes, or lower on the face, closer to the tip of the nose. During the task, observers maintained their gaze at either their own group's mean PFL or at the other group's mean PFL. We found that the top half of the face elicits a stronger CFE than the bottom half. Further, the strength of the CFE was modulated by the distance of the PFL from the eyes, such that individuals with a PFL closer to the eyes had a stronger CFE than those with a PFL closer to the mouth. Finally, the top-half CFE for both upper-lookers and lower-lookers was abolished when they fixated at a non-preferred location on the face. Our findings show that the CFE relies on internal face representations shaped by the long-term use of a consistent oculomotor strategy to view faces.