In everyday life, we often make judgments regarding the sequence of events, for example, deciding whether a baseball runner's foot hit the plate before or after the ball hit the glove. Numerous studies have examined the functional correlates of temporal processing using variations of the temporal order judgment and simultaneity judgment (SJ) tasks. To perform temporal order judgment tasks, observers must bind temporal information with identity and/or spatial information relevant to the task itself. SJs, on the other hand, require observers to detect stimulus asynchrony but not the order of stimulus presentation and represent a purer measure of temporal processing. Some previous studies suggest that these temporal decisions rely primarily on right-hemisphere parietal structures, whereas others provide evidence that temporal perception depends on bilateral TPJ or inferior frontal regions (inferior frontal gyrus). Here, we report brain activity elicited by a visual SJ task. Our methods are unique given our use of two orthogonal control conditions, discrimination of spatial orientation and color, which were used to control for brain activation associated with the classic dorsal ("where/how") and ventral ("what") visual pathways. Our neuroimaging experiment shows that performing the SJ task selectively activated a bilateral network in the parietal (TPJ) and frontal (inferior frontal gyrus) cortices. We argue that SJ tasks are a purer measure of temporal perception because they do not require observers to process either identity or spatial information, both of which may activate separate cognitive networks.