Introduction: The recent developments of magnetic resonance (MR) based adaptive strategies for photon and, potentially for proton therapy, require a fast and reliable conversion of MR images to X-ray computed tomography (CT) values. CT values are needed for photon and proton dose calculation. The improvement of conversion results employing a 3D deep learning approach is evaluated. Material and methods: A database of 89 T1-weighted MR head scans with about 100 slices each, including rigidly registered CTs, was created. Twenty-eight validation patients were randomly sampled, and four patients were selected for application. The remaining patients were used to train a 2D and a 3D U-shaped convolutional neural network (Unet). A stack size of 32 slices was used for 3D training. For all application cases, volumetric modulated arc therapy photon and single-field uniform dose pencil-beam scanning proton plans at four different gantry angles were optimized for a generic target on the CT and recalculated on 2D and 3D Unet-based pseudoCTs. Mean (absolute) error (MAE/ME) and a gradient sharpness estimate were used to quantify the image quality. Three-dimensional gamma and dose difference analyses were performed for photon (gamma criteria: 1%, 1 mm) and proton dose distributions (gamma criteria: 2%, 2 mm). Range (80% fall off) differences for beam's eye view profiles were evaluated for protons. Results: Training 36 h for 1000 epochs in 3D (6 h for 200 epochs in 2D) yielded a maximum MAE of 147 HU (135 HU) for the application patients. Except for one patient gamma pass rates for photon and proton dose distributions were above 96% for both Unets. Slice discontinuities were reduced for 3D training at the cost of sharpness. Conclusions: Image analysis revealed a slight advantage of 2D Unets compared to 3D Unets. Similar dose calculation performance was reached for the 2D and 3D network.