The strongest interaction between microscopic spins in magnetic materials is the exchange interaction Jex. Therefore, ultrafast control of Jex holds the promise to control spins on ultimately fast timescales. We demonstrate that time-periodic modulation of the electronic structure by electric fields can be used to reversibly control Jex on ultrafast timescales in extended antiferromagnetic Mott insulators. In the regime of weak driving strength, we find that Jex can be enhanced and reduced for frequencies below and above the Mott gap, respectively. Moreover, for strong driving strength, even the sign of Jex can be reversed and we show that this causes time reversal of the associated quantum spin dynamics. These results suggest wide applications, not only to control magnetism in condensed matter systems, for example, via the excitation of spin resonances, but also to assess fundamental questions concerning the reversibility of the quantum many-body dynamics in cold atom systems.