Prior studies have reported instances of both intact and impaired working memory (WM) performance in people with autism spectrum disorder (ASD). In order to investigate the relation between autistic traits that extend into the normal population and WM, 104 normal college-aged students who varied in their levels of autistic traits were tested. The loading of ASD-associated traits in the normal population leads to differing predictions about WM performance. ASD traits related to a local processing style (or "attention to detail") might enhance WM while ASD-associated traits related to difficulty switching attention and reorienting focus (or "social interaction") might impair WM performance. To assess these predictions, participants filled out the Autism Spectrum Quotient (AQ) and performed a working memory task with both visual and verbal variants. AQ scores were then broken into "attention to detail" and "social interaction" factors, as proposed by Hoekstra and colleagues. The results showed that AQ scores did not predict verbal WM performance but they did predict visual WM performance. The social interaction and attention to detail factors of the AQ had opposing relationships with visual WM performance: A higher level of social difficulty was associated with significantly poorer visual WM performance while a higher level of attention to detail was associated with enhanced visual WM performance. Further investigation of the relation between AQ and WM using the original five-factor model proposed by Baron-Cohen and colleagues (2001) revealed an association between impoverished imagination and visual WM overall.