In cued task switching, decreasing switch costs with increasing response-to-cue interval (RCI) is a typical finding. The traditional account assumes an underlying process of task-set decay. In contrast, we suggest that these RCI effects are due to the influence of temporal distinctiveness on cue-based task retrieval (or reactivation). The present study explored the task-set components that are affected by temporal distinctiveness. According to the idea that a task set consists of response-related components and stimulus-related components, we manipulated response valence (bivalent vs. univalent) in Experiment 1. The results showed that lengthening the RCI leads to a loss of a task-repetition benefit, mainly when the RCI changed from the previous trial to the current trial, but this data pattern did not depend on response valence. In Experiment 2, stimulus valence was manipulated. The results revealed substantially stronger RCI effects with bivalent stimuli than with univalent stimuli. Taken together, the data are inconsistent with task-set decay accounts and suggest that the influence of RCI in task switching modulates cue-based retrieval of stimulus-related task components, such as biasing parameters.