The Herschel-Bulkley law, a universal constitutive relation, has been empirically known to be applicable to a vast range of soft materials, including sheared glasses. Although the Herschel-Bulkley law has attracted public attention, its structural origin has remained an open question. In this Letter, by means of atomistic simulation of binary Lennard-Jones glasses, we report that the instantaneous normal modes with negative eigenvalues, or so-called imaginary modes, serve as the structural signatures for the Herschel-Bulkley rheology in sheared glasses.