Scale-adaptive deep network for deformable image registration

Purpose: Multiresolution hierarchical strategy is typically used in conventional optimization-based image registration to capture varying magnitudes of deformations while avoiding undesirable local minima. A rough concept of the scale is captured in deep networks by the reception field of kernels, and it has been realized to be both desirable and challenging to capture convolutions of different scales simultaneously in registration networks. In this study, we propose a registration network that is conscious of and self-adaptive to deformation of various scales to improve registration performance.

Methods: Dilated inception modules (DIMs) are proposed to incorporate receptive fields of different sizes in a computationally efficient way. Scale adaptive modules (SAMs) are proposed to guide and adjust shallow features using convolutional kernels with spatially adaptive dilation rate learned from deep features. DIMs and SAMs are integrated into a registration network which takes a U-net structure. The network is trained in an unsupervised setting and completes registration with a single evaluation run.

Results: Experiment with two-dimensional (2D) cardiac MRIs showed that the adaptive dilation rate in SAM corresponded well to the deformation scale. Evaluated with left ventricle segmentation, our method achieved a Dice coefficient of (0.93 ± 0.02), significantly better than SimpleElastix and networks without DIM or SAM. The average surface distance was less than 2 mm, comparable to SimpleElastix without statistical significance. Experiment with synthetic data demonstrated the effectiveness of DIMs and SAMs, which led to a significant reduction in target registration error (TRE) based on dense deformation field. The three-dimensional (3D) version of the network achieved a 2.52 mm mean TRE on anatomical landmarks in DIR-Lab thoracic 4DCTs, lower than SimpleElastix and networks without DIM or SAM with statistical significance. The average registration times were 0.002 s for 2D images with size 256 × 256 and 0.42 s for 3D images with size 256 × 256 × 96.

Conclusions: The introduction and integration of DIMs and SAMs addressed the heterogeneous scale problem in an efficient and self-adaptive way. The proposed method provides an alternative to the inefficient multiresolution registration setups.