Compared with autosomes, the X chromosome shows different patterns of evolution as a result of its hemizygosity in males. Additionally, inactivation of the X during spermatogenesis can make the X chromosome an unfavorable location for male-specific genes. These factors can help to explain why in many species gene content of the X chromosome differs from that of autosomes. Indeed, the X chromosome in mouse is enriched for male-specific genes while they are depleted on the X in Drosophila but show neither of these trends in mosquito. Here, we will discuss recent findings on the ancestral and neo-X chromosomes in Drosophila that support sexual antagonism as a force shaping gene content evolution of sex chromosomes and suggest that selection could be driving male-biased genes off the X.