Aging is associated with structural brain changes, cognitive decline, and neurodegenerative diseases. Brain age, an imaging biomarker sensitive to deviations from healthy aging, offers insights into structural aging variations and is a potential prognostic biomarker in neurodegenerative conditions. This study introduces BrainAgeNeXt, a novel convolutional neural network inspired by the MedNeXt framework, designed to predict brain age from T1-weighted magnetic resonance imaging (MRI) scans. BrainAgeNeXt was trained and validated on 11,574 MRI scans from 33 private and publicly available datasets of healthy volunteers, aged 5 to 95 years, imaged with 3T and 7T MRI. Performance was compared against three state-of-the-art brain age prediction methods. BrainAgeNeXt achieved a mean absolute error (MAE) of 2.78 ± 3.64 years, lower than the compared methods (MAE = 3.55, 3.59, and 4.16 years, respectively). We tested all methods also across different levels of image quality, and BrainAgeNeXt performed well even with motion artifacts and less common 7T MRI data. In three longitudinal multiple sclerosis (MS) cohorts (273 individuals), brain age was, on average, 4.21 ± 6.51 years greater than chronological age. Longitudinal analysis indicated that brain age increased by 1.15 years per chronological year in individuals with MS (95% CI = [1.05, 1.26]). Moreover, in early MS, individuals with worsening disability had a higher annual increase in brain age compared to those with stable clinical assessments (1.24 vs. 0.75, p < 0.01). These findings suggest that brain age is a promising prognostic biomarker for MS progression and potentially a valuable endpoint for clinical trials.