Without rigorous symmetry constraints, solutions to approximate electronic structure methods may artificially break symmetry. In the case of the relativistic electronic structure, if time-reversal symmetry is not enforced in calculations of molecules not subject to a magnetic field, it is possible to artificially break Kramers degeneracy in open shell systems. This leads to a description of excited states that may be qualitatively incorrect. Despite this, different electronic structure methods to incorporate correlation and excited states can partially restore Kramers degeneracy from a broken symmetry solution. For single-reference techniques, the inclusion of double and possibly triple excitations in the ground state provides much of the needed correction. Formally, however, this imbalanced treatment of the Kramers-paired spaces is a multi-reference problem, and so methods such as complete-active-space methods perform much better at recovering much of the correct symmetry by state averaging. Using multi-reference configuration interaction, any additional corrections can be obtained as the solution approaches the full configuration interaction limit. A recently proposed "Kramers contamination" value is also used to assess the magnitude of symmetry breaking.