Anogenital mammary-like glands may give rise to various pathologic lesions identical to those known in mammary pathology. Tumor occurring in the anogenital region is extremely rare. The histogenetic origin of this tumor is controversial as it is being debated whether such lesions evolve from ectopic breast tissue and most recently, anogenital mammary-like gland. We report a 28-year-old girl who presented with a painless mass in the anogenital region, which was subsequently excised. Microscopic examination revealed morphologic pattern characteristic of benign phyllodes tumor with pseudoangiomatous stromal hyperplasia. We present this case to emphasize the importance of recognizing this uncommon lesion occurring at an extremely unusual site. We also discuss the histogenesis of phyllodes tumor and related lesions occurring in the anogenital region in light of the current literature along with a brief review of the previously reported cases of anogenital mammary-like glands.