Sensory experience and spontaneous activity play important roles in development of sensory circuits; however, their relative contributions are unclear. Here, we test the role of different forms of activity on remodeling of the mouse retinogeniculate synapse. We found that the bulk of maturation occurs without patterned sensory activity over 4 days spanning eye opening. During this early developmental period, blockade of spontaneous retinal activity by tetrodotoxin, but not visual deprivation, retarded synaptic strengthening and inhibited pruning of excess retinal afferents. Later in development, synaptic remodeling becomes sensitive to changes in visually evoked activity, but only if there has been previous visual experience. Synaptic strengthening and pruning were disrupted by visual deprivation following 1 week of vision, but not by chronic deprivation from birth. Thus, spontaneous activity is necessary to drive the bulk of synaptic refinement around the time of eye opening, while sensory experience is important for the subsequent maintenance of connections.