It is now recognized that a number of cognitive, behavioral, and mental health outcomes across the lifespan can be traced to fetal development. Although the direct mediation is unknown, the substantial variance in fetal growth, most commonly indexed by birth weight, may affect lifespan brain development. We investigated effects of normal variance in birth weight on MRI-derived measures of brain development in 628 healthy children, adolescents, and young adults in the large-scale multicenter Pediatric Imaging, Neurocognition, and Genetics study. This heterogeneous sample was recruited through geographically dispersed sites in the United States. The influence of birth weight on cortical thickness, surface area, and striatal and total brain volumes was investigated, controlling for variance in age, sex, household income, and genetic ancestry factors. Birth weight was found to exert robust positive effects on regional cortical surface area in multiple regions as well as total brain and caudate volumes. These effects were continuous across birth weight ranges and ages and were not confined to subsets of the sample. The findings show that (i) aspects of later child and adolescent brain development are influenced at birth and (ii) relatively small differences in birth weight across groups and conditions typically compared in neuropsychiatric research (e.g., Attention Deficit Hyperactivity Disorder, schizophrenia, and personality disorders) may influence group differences observed in brain parameters of interest at a later stage in life. These findings should serve to increase our attention to early influences.