Linear spin wave theory (LSWT) is the standard technique to compute the spectra of magnetic excitations in quantum materials. In this Letter, we show that LSWT, even under ordinary circumstances, may fail to implement the symmetries of the underlying ordered magnetic Hamiltonian leading to spurious degeneracies. In common with pseudo-Goldstone modes in cases of quantum order by disorder these degeneracies tend to be lifted by magnon-magnon interactions. We show how, instead, the correct symmetries may be restored at the level of LSWT. In the process we give examples, supported by nonperturbative matrix product based time evolution calculations, where symmetry dictates topological features but where LSWT fails to implement them. We also comment on possible spin split magnons in MnF_{2} and similar rutiles by analogy to recently proposed altermagnets.