Histone lysine methylation is an important epigenetic modification with both activating and repressive roles in gene expression. Jumonji C (jmjC) domain-containing proteins have been shown to reverse histone methylation in nonplant model systems. Here, we show that plant Jumonji C proteins have both conserved and specific features compared with mammalian homologues. In particular, the rice JMJD2 family jmjC gene JMJ706 is shown to encode a heterochromatin-enriched protein. The JMJ706 protein specifically reverses di- and trimethylations of lysine 9 of histone H3 (H3K9) in vitro. Loss-of-function mutations of the gene lead to increased di- and trimethylations of H3K9 and affect the spikelet development, including altered floral morphology and organ number. Gene expression and histone modification analysis indicates that JMJ706 regulates a subset of flower development regulatory genes. Taken together, our data suggest that rice JMJ706 encodes a heterochromatin-associated H3K9 demethylase involved in the regulation of flower development in rice.