Magnetic resonance imaging (MRI) offers outstanding soft tissue contrast that may reduce uncertainties in target and organ-at-risk delineation and enable online adaptive image-guided treatment. Spatial distortions resulting from non-linearities in the gradient fields and non-uniformity in the main magnetic field must be accounted for across the imaging field-of-view to prevent systematic errors during treatment delivery. This work presents a modular phantom and software application to characterize geometric distortion (GD) within the large field-of-view MRI images required for radiation therapy simulation. The modular phantom is assembled from a series of rectangular foam blocks containing high-contrast fiducial markers in a known configuration. The modular phantom design facilitates transportation of the phantom between different MR scanners and MR-guided linear accelerators and allows the phantom to be adapted to fit different sized bores or coils. The phantom was evaluated using a 1.5 T MR-guided linear accelerator (MR-Linac) and 1.5 T and 3.0 T diagnostic scanners. Performance was assessed by varying acquisition parameters to induce image distortions in a known manner. Imaging was performed using T1 and T2 weighted pulse sequences with 2D and 3D distortion correction algorithms and the receiver bandwidth (BW) varied as 250-815 Hz pixel-1. Phantom set-up reproducibility was evaluated across independent set-ups. The software was validated by comparison with a non-modular phantom. Average geometric distortion was 0.94 ± 0.58 mm for the MR-Linac, 0.90 ± 0.53 mm for the 1.5 T scanner, and 1.15 ± 0.62 mm for the 3.0 T scanner, for a 400 mm diameter volume-of-interest. GD increased, as expected, with decreasing BW, and with the 2D versus 3D correction algorithm. Differences in GD attributed to phantom set-up were 0.13 mm or less. Differences in GD for the two software applications were less than 0.07 mm. A novel modular phantom was developed to evaluate distortions in MR images for radiation therapy applications.