The basic plan of the retina is conserved across vertebrates, yet species differ profoundly in their visual needs (Baden et al., 2020). One might expect that retinal cell types evolved to accommodate these varied needs, but this has not been systematically studied. Here, we generated and integrated single-cell transcriptomic atlases of the retina from 17 species: humans, two non-human primates, four rodents, three ungulates, opossum, ferret, tree shrew, a teleost fish, a bird, a reptile and a lamprey. Molecular conservation of the six retinal cell classes (photoreceptors, horizontal cells, bipolar cells, amacrine cells, retinal ganglion cells [RGCs] and Muller glia) is striking, with transcriptomic differences across species correlated with evolutionary distance. Major subclasses are also conserved, whereas variation among types within classes or subclasses is more pronounced. However, an integrative analysis revealed that numerous types are shared across species based on conserved gene expression programs that likely trace back to the common ancestor of jawed vertebrates. The degree of variation among types increases from the outer retina (photoreceptors) to the inner retina (RGCs), suggesting that evolution acts preferentially to shape the retinal output. Finally, we identified mammalian orthologs of midget RGCs, which comprise >80% of RGCs in the human retina, subserve high-acuity vision, and were believed to be primate-specific (Berson, 2008); in contrast, the mouse orthologs comprise <2% of mouse RGCs. Projections both primate and mouse orthologous types are overrepresented in the thalamus, which supplies the primary visual cortex. We suggest that midget RGCs are not primate innovations, but descendants of evolutionarily ancient types that decreased in size and increased in number as primates evolved, thereby facilitating high visual acuity and increased cortical processing of visual information.