Multiple sclerosis (MS) is a multifocal demyelinating disease of the central nervous system, with lesions widespread through the brain and spinal cord. An important manifestation is cognitive impairment, which, though difficult to measure, may have a major social impact. To better understand the relationship between structural tissue damage and cognitive impairment, we examined the extent and spatial distribution of brain lesions, as measured by magnetic resonance imaging (MRI), in relation to abnormal cognitive performance as measured by the Brief Repeatable Battery (BRB) in 82 MS patients. Possible confounders, like fatigue, pain and depression were also assessed. Brain MR image analysis included hyperintense T2 and hypointense T1 lesion load in the whole brain and the four lobes separately, as well as whole brain volume measurements. Cognitive impairment (defined as more than two abnormal tests) was found in 67% of the patients. Moderately strong correlations were found between the subtests of the BRB and the lesion loads in the brain regions hypothesized to be associated with that cognitive test, although these correlations were in general not much stronger than those between the subtests and the overall lesion load (due to strong interrelationships). The Spatial Recall Test correlated best with parietal lesion load; the Symbol Digit Modalities Test, the Paced Auditory Serial Addition Task (PASAT) and the Word List Generation best with frontal, parietal and temporal lesion load; while the Verbal List Generation Test Index correlated only with atrophy. Atrophy and lesion load were the main factors determining the test scores, explaining 10-25% of the variance in the test results, and were more important than fatigue, pain and depression; only depression had a minor, but significant, additional effect on the PASAT. In conclusion, cognitive impairment in MS is moderately dependent on amount (and distribution) of structural brain damage, especially in the more physically impaired patients group.