Chromosome segment duplications are integral in genome evolution by providing a source for the origin of new genes. In the rice genome, besides an ancient polyploidy event known in the rice common ancestor, it had been identified that there was a special segmental duplication involving chromosomes 11 and 12, but the biological role of this duplication remains unknown. In this study, by using a set of chromosome segment substitution lines (CSSLs) and near isogenic lines (NILs) derived from the indica cultivar 9311 and japonica cultivar Nipponbare, a major QTL (qS12) resulting in hybrid male sterility was mapped within ~400 kb region adjacent to the special duplicated segment on the short arm of chromosome 12. Compared to the japonica cultivar Nipponbare, the two sides of the qS12 candidate region were inverted in the indica cultivar 9311. Among 47 of the 111 rice genotypes evaluated by molecular markers, the inverted sides were detected, and found completely homologous to indica cultivar 9311. These results suggested that the two inverted sides protect the sequence in the qS12 regions from recombination. On the short-arm of chromosome 12, two QTLs S-e and S25, in addition to qS12, were previously detected as a distinct segregation distortion and pollen semi-sterility loci. We propose these three hybrid sterility loci are the same locus, and the duplicated segment on chromosome 12 may play a prominent role in diversification, i.e., sub-speciation of cultivated rice.