Face cognition performance is related to individual differences in cognitive subprocesses, as reflected in the amplitudes and latencies of event-related brain potentials (ERPs; Herzmann, Kunina, Sommer, & Wilhelm, 2010). In order to replicate and extend these findings, 110 participants were tested on a comprehensive task battery measuring face cognition abilities and established cognitive abilities, followed by ERP recordings in a face-learning-and-recognition task. We replicated the links of the ERP components indicating the speed of structural face encoding (N170 latency) and access to structural representations in memory (early repetition effect [ERE]/N250r) with the accuracy and speed of face cognition and with established cognitive abilities. As a novel result, we differentiated between the accuracy of face perception and face memory on the behavioral and electrophysiological levels and report a relationship between basic visual processes (P100 amplitude) and face memory. Moreover, the brain-behavior relationships for the ERE/N250r held true, even though we eliminated pictorial and perceptual structural codes from the priming effects by using backward masking of the primes with novel unfamiliar faces. On a methodological level, we demonstrated the utility of the latent difference score modeling technique to parameterize ERP difference components (e.g., ERE/N250r) on a latent level and link them to face cognition abilities.