Large spin-orbit coupling in combination with circular dichroism allows access to spin-polarized and valley-polarized states in a controlled way in transition metal dichalcogenides. The promising application in spin-valleytronics devices requires a thorough understanding of intervalley coupling mechanisms, which determine the lifetime of spin and valley polarizations. Here we present a joint theory-experiment study shedding light on the Dexter-like intervalley coupling. We reveal that this mechanism couples A and B excitonic states in different valleys, giving rise to an efficient intervalley transfer of coherent exciton populations. We demonstrate that the valley polarization vanishes and is even inverted for A excitons, when the B exciton is resonantly excited and vice versa. Our theoretical findings are supported by energy-resolved and valley-resolved pump-probe experiments and also provide an explanation for the recently measured up-conversion in photoluminescence. The gained insights might help to develop strategies to overcome the intrinsic limit for spin and valley polarizations.