The objective of this review is to discuss the less common causes of rectal and perirectal disease with an emphasis on their differentiating radiological features and the importance of a multimodality imaging approach. Radiologists should be aware of the ultrasound, computed tomography, and magnetic resonance imaging appearances of the wide variety of uncommon lesions arising from the rectal and perirectal regions that may simulate adenocarcinoma in order to render an accurate diagnosis and facilitate appropriate management.