We propose heterostructures that realize first and second order topological superconductivity with vanishing net magnetization by utilizing altermagnetism. Such platforms may offer a significant improvement over conventional platforms with uniform magnetization since the latter suppresses the superconducting gap. We first introduce a 1D semiconductor-superconductor structure in proximity to an altermagnet that realizes end Majorana zero modes (MZMs) with vanishing net magnetization. Additionally, a coexisting Zeeman term provides a tuning knob to distinguish topological and trivial zero modes. We then propose 2D altermagnetic platforms that can realize chiral Majorana fermions or higher order corner MZMs. Our Letter paves the way toward realizing Majorana boundary states with an alternative source of time-reversal breaking and zero net magnetization.