Adaptation to specific visuomotor mappings becomes faster when switching back and forth between them. What is learned when repeatedly switching between the visuomotor mappings: the absolute mappings or the relative shift between the mappings? To test this, we trained participants in a rapid pointing task using a unique color cue as context for each mapping between pointing location and visual feedback. After extensive training, participants adapted to a new mapping using a neutral contextual cue. For catch trials (a change in cue and no visual feedback) different adaptation performances are predicted depending on how the mappings are encoded. When encoding an absolute mapping for each cue, participants would fall back to the mapping associated with the cue irrespective of the state they are currently in. In contrast, when a shift in mapping is encoded for the cue, pointing performance will shift relative to the current mapping by an amount equal to the difference between the previously learned mappings. Results indicate that the contextual cues signal absolute visuomotor mappings rather than relative shifts between mappings.