Bone morphogenetic proteins (BMPs) belong to the TGF-β superfamily of structurally related signaling proteins that regulate a wide array of cellular functions. The key step in BMP signal transduction is the BMP receptor-mediated phosphorylation of transcription factors Smad1, 5, and 8 (collectively Smad1/5/8), which leads to the subsequent activation of BMP-induced gene transcription in the nucleus. In this study, we describe the identification and characterization of PPM1H as a novel cytoplasm-localized Smad1/5/8-specific phosphatase. PPM1H directly interacts with Smad1/5/8 through its Smad-binding domain, and dephosphorylates phospho-Smad1/5/8 (P-Smad1/5/8) in the cytoplasm. Ectopic expression of PPM1H attenuates BMP signaling, whereas loss of PPM1H activity or expression greatly enhances BMP-dependent gene regulation and mesenchymal differentiation. In conclusion, this study suggests that PPM1H acts as a gatekeeper to prevent excessive BMP signaling through dephosphorylation and subsequent nuclear exclusion of P-Smad1/5/8 proteins.