The formation and stability of synapses are key questions in neuroscience. Post-synaptic domains have been classically conceived as resulting from local insertion and turnover of proteins at the synapse. However, insertion is likely to occur outside the post-synaptic domains and advances in single-molecule imaging have shown that proteins diffuse in the plane of the membrane prior to their accumulation at synapses. We quantitatively investigated this scenario using computer simulations and mathematical analysis, taking for definiteness the specific case of inhibitory synapse components, i.e., the glycine receptor (GlyR) and the associated gephyrin scaffolding protein. The observed domain sizes of scaffold clusters can be explained by a dynamic balance between the aggregation of gephyrin proteins diffusing while bound to GlyR and their turnover at the neuron membrane. We also predict the existence of extrasynaptic clusters with a characteristic size distribution that significantly contribute to the size fluctuations of synaptic domains. New super-resolution data for gephyrin proteins established the existence of extrasynaptic clusters the sizes of which are consistent with the model predictions in a range of model parameters. At a general level, our results highlight aggregation with removal as a non-equilibrium phase separation which produces structures of tunable size.