The quality of mother-infant interaction during the first year may be hampered by maternal substance abuse and co-existing non-optimal factors such as psychiatric problems and difficult relational experiences. In the present study three groups of women were recruited during pregnancy: One group with substance abuse problems from residential treatment centers (n = 28), a second group from psychiatric outpatient treatment centers (n = 22), and a third group from well-baby clinics (n = 30). Four maternal optimality indexes were assessed (substance abuse, psychiatric problems, relational experiences and SES). Mother-infant interaction was observed at 3 and 12 months. The substance abuse group showed the most disturbed mother-infant interaction at 12 months. Low maternal optimality as well as impairments in maternal affective involvement at 3 months influenced negatively both on infant and dyadic affective behavior in interaction at 12 months. Long-term interventions are needed to promote affective reciprocity among mother-baby pairs with low optimality.