This study examined differences between cocaine and non-cocaine-using mothers, and between parental and non-parental caregivers of cocaine-exposed infants on caregiver childhood trauma, psychiatric symptoms, demographic, and perinatal risks. Participants included 115 cocaine and 105 non-cocaine mother-infant dyads recruited at delivery. Approximately 19% of cocaine mothers lost custody of their infants by 1 month of infant age compared to 0.02% of non-cocaine mothers. Mothers who used cocaine during pregnancy had higher demographic and obstetric risks. Their infants had higher perinatal risks. Birth mothers who retained custody of their infants had higher demographic risks and perinatal risks, higher childhood trauma, and higher psychiatric symptoms compared to birth mothers who did not use cocaine and non-parental caregivers of cocaine-exposed infants. Results highlight the importance of addressing childhood trauma issues and current psychiatric symptoms in substance abuse treatment with women who engaged in substance use during pregnancy.