A noise reduction method based on the block-matching and 4D (BM4D) scheme is proposed to improve the signal-to-noise ratio (SNR) in distributed vibration sensing (DVS) systems. In the proposed scheme, the original Rayleigh backscattering (RBS) signal is converted into a three-dimensional image containing Rayleigh trajectory and energy information. The correlation between the time-domain and spatial-domain signals is then used to achieve the denoising operation. An experimental demonstration containing both one and two vibration points is conducted to verify the effectiveness of the proposed denoising scheme. The experimental results show that the BM4D scheme can provide higher SNR improvement than the current normalized least mean square (NLM), empirical mode decomposition combined with time-frequency peak filtering (EMD-TFPF), and BM3D schemes. Based on the BM4D scheme, the SNR is improved from 1.27 dB to 12.84 dB in the condition of one vibration point and from 6.23 dB to 20.14 dB in the condition of two vibration points. It is also indicated that the high-frequency noise of the vibration waveform after the denoising operation is mitigated by more than 30 dB, showing the potential for applications of accurate waveform characterization in cost-effective DVS systems.