Eight, early bilingual, sign language interpreters participated in a PET study, which compared working memory for Swedish Sign Language (SSL) with working memory for audiovisual Swedish speech. The interaction between language modality and memory task was manipulated in a within-subjects design. Overall, the results show a previously undocumented, language modality-specific working memory neural architecture for SSL, which relies on a network of bilateral temporal, bilateral parietal and left premotor activation. In addition, differential activation in the right cerebellum was found for the two language modalities. Similarities across language modality are found in Broca's area for all tasks and in the anterior left inferior frontal lobe for semantic retrieval. The bilateral parietal activation pattern for sign language bears similarity to neural activity during, e.g., nonverbal visuospatial tasks, and it is argued that this may reflect generation of a virtual spatial array. Aspects of the data suggesting an age of acquisition effect are also considered. Furthermore, it is discussed why the pattern of parietal activation cannot be explained by factors relating to perception, production or recoding of signs, or to task difficulty. The results are generally compatible with Wilson's [Psychon. Bull. Rev. 8 (2001) 44] account of working memory.