The aim of the present study was to investigate the role of epigenetic inactivation of the runt-related transcription factor 3 gene (RUNX3) in the malignant transformation of ovarian endometriosis. Samples obtained by microdissection and scraping included 30 malignant ovarian endometriotic cyst tissues and 30 corresponding eutopic endometrium tissues from the endometriosis-associated ovarian carcinoma (EAOC) group, 19 benign ovarian endometriotic cyst tissues and 22 corresponding eutopic endometrium tissues from the endometriosis (EM) group and 22 normal eutopic endometrium tissues from the control endometrium (CE) group. RUNX3 methylation status was determined by methylation-specific PCR and bisulfite sequencing, while levels of RUNX3 and ERα protein expression were evaluated using immunohistochemistry. The percentage of RUNX3 methylation and negative RUNX3 protein expression in the malignant ovarian endometriotic cysts from the EAOC group was significantly higher than that in the benign ovarian endometriotic cysts from the EM group. The percentage of RUNX3 methylation and negative RUNX3 protein expression in the eutopic endometrium from the EAOC group was significantly higher than that in the EM and CE groups. An inverse correlation between positive RUNX3 protein expression and methylation was observed and a positive correlation was shown between RUNX3 methylation and ERα protein expression. In the malignant ovarian endometriotic cysts from the EAOC group, there was no significant correlation between methylation frequency of the RUNX3 gene and histological type. However, the percentage of RUNX3 gene methylation was significantly higher in the tissue samples from patients with surgical stage IC EAOC than the percentage in patients with stage IA and IB disease. These results suggest that RUNX3 inactivation by promoter hypermethylation plays a role in the progression of malignant transformation of ovarian EM and is closely related to estrogen metabolism. Negative protein expression and abnormal RUNX3 methylation in the eutopic endometrium could be used as diagnostic markers in patients with ovarian EM who may be at an increased risk of developing EAOC.