Sex steroids exert important organizational effects on brain structure. Early in life, they are involved in brain sexual differentiation. During puberty, sex steroid levels increase considerably. However, to which extent sex steroid production is involved in structural brain development during human puberty remains unknown. The relationship between pubertal rises in testosterone and estradiol levels and brain structure was assessed in 37 boys and 41 girls (10-15 years). Global brain volumes were measured using volumetric-MRI. Regional gray and white matter were quantified with voxel-based morphometry (VBM), a technique which measures relative concentrations ('density') of gray and white matter after individual global differences in size and shape of brains have been removed. Results showed that, corrected for age, global gray matter volume was negatively associated with estradiol levels in girls, and positively with testosterone levels in boys. Regionally, a higher estradiol level in girls was associated with decreases within prefrontal, parietal and middle temporal areas (corrected for age), and with increases in middle frontal-, inferior temporal- and middle occipital gyri. In boys, estradiol and testosterone levels were not related to regional brain structures, nor were testosterone levels in girls. Pubertal sex steroid levels could not explain regional sex differences in regional gray matter density. Boys were significantly younger than girls, which may explain part of the results. In conclusion, in girls, with the progression of puberty, gray matter development is at least in part directly associated with increased levels of estradiol, whereas in boys, who are in a less advanced pubertal stage, such steroid-related development could not (yet) be found. We suggest that in pubertal girls, estradiol may be implicated in neuronal changes in the cerebral cortex during this important period of brain development.