Multiscale diffractive U-Net: a robust all-optical deep learning framework modeled with sampling and skip connections

As an all-optical learning framework, diffractive deep neural networks (D²NNs) have great potential in running speed, data throughput, and energy consumption. The depth of networks and the misalignment of layers are two problems to limit its further development. In this work, a robust all-optical network framework (multiscale diffractive U-Net, MDUNet) based on multi-scale features fusion has been proposed. The depth expansion and alignment robustness of the network can be significantly improved by introducing sampling and skip connections. Compared with common all-optical learning frameworks, MDUNet achieves the highest accuracy of 98.81% and 89.11% on MNIST and Fashion-MNIST respectively. The testing accuracy of MNIST and Fashion-MNIST can be further improved to 99.06% and 89.86% respectively by using the ensemble learning method to construct the optoelectronic hybrid neural network.