As we perform daily activities--driving to work, unlocking the office door, or grabbing a coffee cup--our actions seem automatic and preprogrammed. Nonetheless, routine, well-practiced behavior is continually modulated by incidental experience: In repetitive experimental tasks, recent (~4) trials reliably influence performance and action choice. Psychological theories downplay the significance of sequential effects, explaining them as rapidly decaying perturbations of behavior, with no long-term consequences. We challenged this traditional perspective in two experiments designed to probe the impact of more distant experience, finding evidence for effects spanning up to a thousand intermediate trials. We present a normative theory in which these persistent effects reflect optimal adaptation to a dynamic environment exhibiting varying rates of change. The theory predicts a heavy-tailed decaying influence of past experience, consistent with our data, and suggests that individual incidental experiences are catalogued in a temporally extended memory utilized in order to optimize subsequent behavior.