The oligomeric amyloid-β (Aβ) peptide is thought to contribute to the subtle amnesic changes in Alzheimer's disease (AD) by causing synaptic dysfunction. Here, we examined the time course of synaptic changes in mouse hippocampal neurons following exposure to Aβ42 at picomolar concentrations, mimicking its physiological levels in the brain. We found opposite effects of the peptide with short exposures in the range of minutes enhancing synaptic plasticity, and longer exposures lasting several hours reducing it. The plasticity reduction was concomitant with an increase in the basal frequency of spontaneous neurotransmitter release, a higher basal number of functional presynaptic release sites, and a redistribution of synaptic proteins including the vesicle-associated proteins synapsin I, synaptophysin, and the post-synaptic glutamate receptor I. These synaptic alterations were mediated by cytoskeletal changes involving actin polymerization and p38 mitogen-activated protein kinase. These in vitro findings were confirmed in vivo with short hippocampal infusions of picomolar Aβ enhancing contextual memory and prolonged infusions impairing it. Our findings provide a model for initiation of synaptic dysfunction whereby exposure to physiologic levels of Aβ for a prolonged period of time causes microstructural changes at the synapse which result in increased transmitter release, failure of synaptic plasticity, and memory loss.