PARP inhibitors have gained attention in the treatment of metastatic castration-resistant prostate cancer, but approximately half of patients have to abort treatment due to severe hematological toxicity. Herein, we proposed a prostate-specific membrane antigen (PSMA)-targeting small molecule-drug conjugate (SMDC) strategy to address this issue. This led to CQ-16, which achieved its targeting to prostate tumor cells through binding to PSMA. Also, CQ-16 retained the PARP inhibitory activity and exhibited highly selective antiproliferative activities between PSMA-positive and PSMA-negative prostate cells. Moreover, the hematological toxicity observed in Olaparib was not showing in the group of CQ-16 even at a high dose of 390 mg/kg. Moreover, oral administration of CQ-16 exerted significant tumor growth inhibition in the 22Rv1 xenograft mouse model. These above findings not only highlight the potential of CQ-16 to overcome the hematological toxicity associated with PARP inhibitors but also provide a strategy to develop an SMDC with enhanced safety profiles.