ARID4A and ARID4B are homologous members of the ARID (AT-rich interaction domain) gene family. ARID4A and ARID4B physically interact with each other. ARID4A is a retinoblastoma (RB)-binding protein. Biological function of these interactions is still unknown. Here, we report that mice with complete deficiency of Arid4a, combined with haploinsufficiency of Arid4b (Arid4a(-/-)Arid4b(+/-)), showed progressive loss of male fertility, accompanied by hypogonadism and seminal vesicle agenesis/hypodysplasia. Arid4a and Arid4b are expressed mainly in Sertoli cells of testes, which implies that their roles in Sertoli cell function are to support spermatogenesis and create the impermeable blood-testis barrier. In fact, evaluation of germ cell development in the Arid4a(-/-)Arid4b(+/-) mice showed spermatogenic arrest at the stages of meiotic spermatocytes and postmeiotic haploid spermatids. Analysis of the integrity of the blood-testis barrier showed increased permeability of seminiferous tubules in the Arid4a(-/-)Arid4b(+/-) testes. Interestingly, phenotypic Sertoli cell dysfunction in the Arid4a(-/-)Arid4b(+/-) mice, including spermatogenic failures and the impaired blood-testis barrier, recapitulated the defects found in the Sertoli cell-specific androgen receptor (AR) knockout mice and the Sertoli cell-specific RB knockout mice. Investigation of the molecular mechanism identified several AR- and RB-responsive genes as downstream targets of ARID4A and ARID4B. Our results thus indicate that ARID4A and ARID4B function as transcriptional coactivators for AR and RB and play an integral part in the AR and RB regulatory pathways involved in the regulation of Sertoli cell function and male fertility.