DNA methylation is an important epigenetic mechanism essential for transposon silencing and genome integrity. Across evolution, the substrates of DNA methylation have diversified between kingdoms. In plants, chromomethylase3 (CMT3) and CMT2 mediate CHG and CHH methylation, respectively. However, how these two methyltransferases diverge on substrate specificities during evolution remains unknown. Here, we reveal that CMT2 originates from a duplication of an evolutionarily ancient CMT3 in flowering plants. Lacking a key arginine residue recognizing CHG in CMT2 impairs its CHG methylation activity in most flowering plants. An engineered V1200R mutation empowers CMT2 to restore CHG and CHH methylations in Arabidopsis cmt2cmt3 mutant, testifying a loss-of-function effect for CMT2 during evolution. CMT2 has evolved a long and unstructured amino terminus critical for protein stability, especially under heat stress, and is plastic to tolerate various natural mutations. Together, this study reveals the mechanism of chromomethylase divergence for context-specific DNA methylation in plants and sheds important lights on DNA methylation evolution and function.