Femtosecond magneto-optical pump-probe measurements of ultrafast demagnetization show an intriguing difference in the first 100 fs of the magneto-optical Kerr response depending on whether the polarization of the pump and probe beams are in parallel or perpendicular configuration (Bigot et al 2009 Nat. Phys. 5 515). Starting from a most general relativistic Hamiltonian we focus on the ultra-relativistic light-spin interaction and show that this coupling term leads to different light-induced opto-magnetic fields when pump and probe polarization are parallel and perpendicular to each other, providing thus an explanation for the measurements. We also analyze other pump-probe configurations where the pump laser is circularly polarized and the employed probe contains only linearly polarized light and show that similar opto-magnetic effects can be anticipated.