White matter (WM) integrity in the medial temporal lobes and episodic memory performance was examined in patients with mild cognitive impairment (MCI) and age-matched cognitively intact controls. Material specific associations between WM in the left versus right hemisphere and verbal versus visual memory performance were examined as well. Fourteen right-handed amnestic MCI patients underwent diffusion tensor imaging (DTI) and received verbal (words, story) and visual (designs) memory tests. Delayed verbal memory was significantly correlated with loss of WM integrity in the medial temporal lobe. This finding was associated with both the left and right temporal regions. Immediate visual memory performance was significantly correlated with the loss of WM integrity in the left temporal region. The results indicate that WM integrity in the medial temporal lobe is associated with objective memory functioning in MCI. However, strong material specific relationships were not observed, possibly reflecting diverse encoding strategies used by participants such as imagery of verbal material and verbal encoding of designs.