Modulation is a cornerstone of optical communication, and as such, governs the overall speed of data transmission. Currently, the two main strategies for modulating light are direct modulation of the excited emitter population (for example, using semiconductor lasers) and external optical modulation (for example, using Mach-Zehnder interferometers or ring resonators). However, recent advances in nanophotonics offer an alternative approach to control spontaneous emission through modifications to the local density of optical states. Here, by leveraging the phase-change of a vanadium dioxide nanolayer, we demonstrate broadband all-optical direct modulation of 1.5 μm emission from trivalent erbium ions more than three orders of magnitude faster than their excited state lifetime. This proof-of-concept demonstration shows how integration with phase-change materials can transform widespread phosphorescent materials into high-speed optical sources that can be integrated in monolithic nanoscale devices for both free-space and on-chip communication.