For the relativistic Kelvin-Helmholtz instability (KHI), which occurs at shear interfaces between two plasma streams, we report results on the polarized radiation over all observation directions and frequencies emitted by the plasma electrons from ab initio kinetic simulations. We find the polarization of the radiation to provide a clear signature for distinguishing the linear phase of the KHI from its other phases. During the linear phase, we predict the growth rate of the KHI radiation power to match the growth rate of the KHI to a high degree. Our predictions are based on a model of the vortex dynamics, which describes the electron motion in the vicinity of the shear interface between the two streams. Albeit the complex and turbulent dynamics happening in the shear region, we find excellent agreement between our model and large-scale particle-in-cell simulations. Our findings pave the way for identifying the KHI linear regime and for measuring its growth rate in astrophysical jets observable on earth as well as in laboratory plasmas.