Trafficking of AMPA receptors in and out of synapses is crucial for synaptic plasticity. Previous studies have focused on the role of endo/exocytosis processes or that of lateral diffusion of extra-synaptic receptors. We have now directly imaged AMPAR movements inside and outside synapses of live neurons using single-molecule fluorescence microscopy. Inside individual synapses, we found immobile and mobile receptors, which display restricted diffusion. Extra-synaptic receptors display free diffusion. Receptors could also exchange between these membrane compartments through lateral diffusion. Glutamate application increased both receptor mobility inside synapses and the fraction of mobile receptors present in a juxtasynaptic region. Block of inhibitory transmission to favor excitatory synaptic activity induced a transient increase in the fraction of mobile receptors and a decrease in the proportion of juxtasynaptic receptors. Altogether, our data show that rapid exchange of receptors between a synaptic and extra-synaptic localization occurs through regulation of receptor diffusion inside synapses.