Atomic rearrangements induced by shear stress are fundamental for understanding deformation mechanisms in metallic glasses (MGs). Using molecular dynamic simulation, the atomic rearrangements characterized by nonaffine displacements (NADs) and their spatial distribution and evolution with tensile stress in Cu50Zr50 MG were investigated. It was found that in the elastic regime the atomic rearrangements with the largest NADs are relatively homogeneous in space, but exhibit strong spatial correlation, become localized and inhomogeneous, and form large clusters as strain increases, which may facilitate the so-called shear transformation zones. Furthermore, initially they prefer to take place around Cu atoms which have more nonicosahedral configurations. As strain increases, the preference decays and disappears in the plastic regime. The atomic rearrangements with the smallest NADs are preferentially located around Cu atoms, too, but with more icosahedral or icosahedral-like atomic configurations. The preference is maintained in the whole deformation process. In contrast, the atomic rearrangements with moderate NADs distribute homogeneously, and do not show explicit preference or spatial correlation, acting as matrix during deformation. Among the atomic rearrangements with different NADs, those with largest and smallest NADs are nearest neighbors initially, but separating with increasing strain, while those with largest and moderate NADs always avoid to each other. The correlations in the fluctuations of the NADs confirm the long-range strain correlation and the scale-free characteristic of NADs in both elastic and plastic deformation, which suggests a universality of the scaling in the plastic flow in MGs.