We generalize the two-temperature model [Sanders and Walton, Phys. Rev. B 15, 1489 (1977)] for coupled phonon-magnon diffusion to include the effect of the concurrent magnetization flow, with a particular emphasis on the thermal consequence of the magnon flow driven by a nonuniform magnetic field. Working within the framework of the Boltzmann transport equation, we derive the constitutive equations for coupled phonon-magnon transport driven by gradients of both temperature and external magnetic fields, and the corresponding conservation laws. Our equations reduce to the original Sanders-Walton two-temperature model under a uniform external field, but predict a new magnon cooling effect driven by a nonuniform magnetic field in a homogeneous single-domain ferromagnet. We estimate the magnitude of the cooling effect in an yttrium iron garnet, and show it is within current experimental reach. With properly optimized materials, the predicted cooling effect can potentially supplement the conventional magnetocaloric effect in cryogenic applications in the future.