Wolfram Syndrome (WFS) is a rare autosomal recessive disease characterized by insulin-dependent diabetes mellitus, optic nerve atrophy, diabetes insipidus, deafness, and neurological dysfunction leading to death in mid-adulthood. WFS is caused by mutations in the WFS1 gene, which lead to endoplasmic reticulum (ER) stress-mediated cell death. Case studies have found widespread brain atrophy in late stage WFS. However, it is not known when in the disease course these brain abnormalities arise, and whether there is differential vulnerability across brain regions and tissue classes. To address this limitation, we quantified regional brain abnormalities across multiple imaging modalities in a cohort of young patients in relatively early stages of WFS. Children and young adults with WFS were evaluated with neurological, cognitive and structural magnetic resonance imaging measures. Compared to normative data, the WFS group had intact cognition, significant anxiety and depression, and gait abnormalities. Compared to healthy and type 1 diabetic control groups, the WFS group had smaller intracranial volume and preferentially affected gray matter volume and white matter microstructural integrity in the brainstem, cerebellum and optic radiations. Abnormalities were detected in even the youngest patients with mildest symptoms, and some measures did not follow the typical age-dependent developmental trajectory. These results establish that WFS is associated with smaller intracranial volume with specific abnormalities in the brainstem and cerebellum, even at the earliest stage of clinical symptoms. This pattern of abnormalities suggests that WFS has a pronounced impact on early brain development in addition to later neurodegenerative effects, representing a significant new insight into the WFS disease process. Longitudinal studies will be critical for confirming and expanding our understanding of the impact of ER stress dysregulation on brain development.