Sex hormones have been shown to contribute to the organization and function of the brain during puberty and adolescence. Moreover, it has been suggested that distinct hormone changes in girls versus boys may contribute to the emergence of sex differences in internalizing and externalizing behavior during adolescence. In the current longitudinal study, the influence of within-subject changes in puberty (physical and hormonal) on cortical thickness and surface area was examined across a 2-year span, while controlling for age. Greater increases in Tanner Stage predicted less superior frontal thinning and decreases in precuneus surface area in both sexes. Significant Tanner Stage and sex interactions were also seen, with less right superior temporal thinning in girls but not boys, as well as greater decreases in the right bank of the superior temporal sulcus surface area in boys compared to girls. In addition, within-subject changes in testosterone over the 2-year follow-up period were found to relate to decreases in middle superior frontal surface area in boys, but increases in surface area in girls. Lastly, larger increases in estradiol in girls predicted greater middle temporal lobe thinning. These results show that within-subject physical and hormonal markers of puberty relate to region and sex-specific changes in cortical development across adolescence.