The functional organization of the human auditory cortex is still not well understood with respect to speech perception and language lateralization. Especially, there is comparatively little data available in the brain imaging literature focusing on the timing of phonetic processing. We recorded auditory-evoked potentials (AEP) from 27 scalp and additional EOG channels in 12 healthy volunteers performing a free report dichotic listening task with simple speech sounds (CV syllables: [ba], [da], [ga], [pa], [ta], [ka]). ERP analysis employed independent components analysis (ICA) wavelet denoising for artifact reduction and improvement of the SNR. The main finding was a 15-ms shorter average latency of the N1-AEP recorded from the scalp approximately overlying the left supratemporal cortical plane compared to the N1-AEP over the homologous right side. Corresponding N1 amplitudes did not differ between these sites. The individual AEP latency differences significantly correlated with the ear advantage as an index of speech/language lateralization. The behaviorally relevant difference in N1 latency between the hemispheres indicates that an important key to understanding speech perception is to consider the functional implications of neuronal event timing.