Elastic image registration is widely used to adapt brain images to a common template space, and, in complementary fashion, to adapt an anatomical template to a subject's anatomy. Although HAMMER is a very accurate image-registration algorithm, it requires a 3-class segmentation step prior to registration, and its performance is affected by segmentation quality. We here propose a new framework to improve this algorithm's robustness to poor initial segmentation. Our new framework is based on Adaptive Generalized Expectation Maximization (AGEM) for unified segmentation and registration, in which we use an adaptive strategy to incorporate spatial information from a probabilistic atlas to improve segmentation and registration simultaneously. Our experiments using real MR brain images indicate that our integrated approach improves registration accuracy; we have also found that our iterative approach renders HAMMER robust to low tissue contrast, which hinders 3-class segmentation.