In this Letter, we demonstrate for the first time, to our knowledge, a holographic data synthesis based on a deep learning probabilistic diffusion model (DDPM). Several different datasets of color images corresponding to different types of objects are converted to complex-valued holographic data through backpropagation. Then, we train a DDPM using the resulting holographic datasets. The diffusion model is composed of a noise scheduler, which gradually adds Gaussian noise to each hologram in the dataset, and a U-Net convolutional neural network that is trained to reverse this process. Once the U-Net is trained, any number of holograms with similar features as those of the datasets can be generated just by inputting a Gaussian random noise to the model. We demonstrate the synthesis of holograms containing color images of 2D characters, vehicles, and 3D scenes with different characters at different propagation distances.