Vertical-cavity surface-emitting lasers subjected to weak polarization-insensitive optical feedback are studied experimentally and theoretically. We find that the feedback induces random anticorrelated hopping between the two orthogonal linearly polarized modes. This polarization mode hopping is accompanied by rapid anticorrelated oscillations in the linearly polarized intensities at the external-cavity frequency. The study of a simple stochastic delay differential equation suggests that these oscillations generated by the delay are typical of any hopping phenomenon between states.