Chromatin modifications, especially histone-tail acetylation, have been implicated in memory formation. Increased histone-tail acetylation induced by inhibitors of histone deacetylases (HDACis) facilitates learning and memory in wild-type mice as well as in mouse models of neurodegeneration. Harnessing the therapeutic potential of HDACis requires knowledge of the specific HDAC family member(s) linked to cognitive enhancement. Here we show that neuron-specific overexpression of HDAC2, but not that of HDAC1, decreased dendritic spine density, synapse number, synaptic plasticity and memory formation. Conversely, Hdac2 deficiency resulted in increased synapse number and memory facilitation, similar to chronic treatment with HDACis in mice. Notably, reduced synapse number and learning impairment of HDAC2-overexpressing mice were ameliorated by chronic treatment with HDACis. Correspondingly, treatment with HDACis failed to further facilitate memory formation in Hdac2-deficient mice. Furthermore, analysis of promoter occupancy revealed an association of HDAC2 with the promoters of genes implicated in synaptic plasticity and memory formation. Taken together, our results suggest that HDAC2 functions in modulating synaptic plasticity and long-lasting changes of neural circuits, which in turn negatively regulates learning and memory. These observations encourage the development and testing of HDAC2-selective inhibitors for human diseases associated with memory impairment.