In five experiments, we examined the influence of contextual objects' location and visual features on visual memory. Participants' visual memory was tested with a change detection task in which they had to judge whether the orientation (Experiments 1A, 1B, and 2) or color (Experiments 3A and 3B) of a target object was the same. Furthermore, contextual objects' locations and visual features were manipulated in the test image. The results showed that change detection performance was better when contextual objects' locations remained the same from study to test, demonstrating that the original spatial configuration is important for subsequent visual memory retrieval. The results further showed that changes to contextual objects' orientation, but not color, reduced orientation change detection performance; and changes to contextual objects' color, but not orientation, impaired color change detection performance. Therefore, contextual objects' visual features are capable of affecting visual memory. However, selective attention plays an influential role in modulating such effects.