Patients are increasingly presented with their health data through patient portals in an attempt to engage patients in their own care. Due to the large amounts of data generated during a patient visit, the medical information when shared with patients can be overwhelming and cause anxiety due to lack of understanding. Health care organizations are attempting to improve transparency by providing patients with access to visit information. In this paper, we present our findings from a research study to evaluate patient understanding of medical images. We used cognitive fit theory to evaluate existing tools and images that are shared with patients and analyzed the relevance of such sharing. We discover that medical images need a lot of customization before they can be shared with patients. We suggest that new tools for medical imaging should be developed to fit the cognitive abilities of patients.