Janus transition metal dichalcogenides (TMDs) have recently emerged as a new class of two-dimensional materials with a vertical dipole moment. Here, using time-domain ab initio simulations, we show that electron-hole recombination can be substantially suppressed via different stacking orientations of bilayer MoSSe. Despite having a larger net dipole moment, a S-Se/S-Se oriented MoSSe bilayer has a shorter carrier lifetime due to strong nonadiabatic coupling and a small band gap. The electron-hole recombination is coupled to the interlayer out-of-plane motion. In contrast, the opposite vertical dipoles weaken interlayer interactions in symmetric oriented MoSSe bilayers. Consequently, initial and final states are localized within different layers, and this significantly suppresses carrier recombination, resulting in an order of magnitude longer excited carrier lifetime in Se-S/S-Se oriented MoSSe bilayers. Our simulations provide theoretical insights into the carrier dynamics and suggest a way to enhance the carrier lifetime in Janus TMDs for efficient energy harvesting.