Atoh1 is a basic helix-loop-helix transcription factor necessary for the specification of inner ear hair cells and central auditory system neurons derived from the rhombic lip. We used the Cre-loxP system and two Cre-driver lines (Egr2(Cre) and Hoxb1(Cre)) to delete Atoh1 from different regions of the cochlear nucleus (CN) and accessory auditory nuclei (AAN). Adult Atoh1-conditional knock-out mice (Atoh1(CKO)) are behaviorally deaf, have diminished auditory brainstem evoked responses, and have disrupted CN and AAN morphology and connectivity. In addition, Egr2; Atoh1(CKO) mice lose spiral ganglion neurons in the cochlea and AAN neurons during the first 3 d of life, revealing a novel critical period in the development of these neurons. These new mouse models of predominantly central deafness illuminate the importance of the CN for support of a subset of peripheral and central auditory neurons.