Rod/cone photoreceptors of the outer retina and the melanopsin-expressing retinal ganglion cells (mRGCs) of the inner retina mediate non-image forming visual responses including entrainment of the circadian clock to the ambient light, the pupillary light reflex (PLR), and light modulation of activity. Targeted deletion of the melanopsin gene attenuates these adaptive responses with no apparent change in the development and morphology of the mRGCs. Comprehensive identification of mRGCs and knowledge of their specific roles in image-forming and non-image forming photoresponses are currently lacking. We used a Cre-dependent GFP expression strategy in mice to genetically label the mRGCs. This revealed that only a subset of mRGCs express enough immunocytochemically detectable levels of melanopsin. We also used a Cre-inducible diphtheria toxin receptor (iDTR) expression approach to express the DTR in mRGCs. mRGCs develop normally, but can be acutely ablated upon diphtheria toxin administration. The mRGC-ablated mice exhibited normal outer retinal function. However, they completely lacked non-image forming visual responses such as circadian photoentrainment, light modulation of activity, and PLR. These results point to the mRGCs as the site of functional integration of the rod/cone and melanopsin phototransduction pathways and as the primary anatomical site for the divergence of image-forming and non-image forming photoresponses in mammals.