Computational ghost imaging (CGI) using stereo vision is able to achieve three-dimensional (3D) imaging by using multiple projection units or multiple bucket detectors which are separated spatially. We present a compact 3D CGI system that consists of Risley prisms, a stationary projection unit and a bucket detector. By rotating double prisms to various angles, speckle patterns appear to be projected by a dynamic virtual projection unit at different positions and multi-view ghost images are obtained for 3D imaging. In the process of reconstruction, a convolutional neural network (CNN) for super-resolution (SR) is adopted to enhance the angular resolution of reconstructed images. Moreover, an optimized 3D CNN is implemented for disparity estimation and 3D reconstruction. The experimental results validate the effectiveness of the method and indicate that the compact system with flexibility has potential in applications such as navigation and detection.