We propose robust and efficient tests and estimators for gene-environment/gene-drug interactions in family-based association studies in which haplotypes, dichotomous/quantitative phenotypes, and complex exposure/treatment variables are analyzed. Using causal inference methodology, we show that the tests and estimators are robust against unmeasured confounding due to population admixture and stratification, provided that Mendel's law of segregation holds and that the considered exposure/treatment variable is not affected by the candidate gene under study. We illustrate the practical relevance of our approach by an application to a chronic obstructive pulmonary disease study. The data analysis suggests a gene-environment interaction between a single nucleotide polymorphism in the Serpine2 gene and smoking status/pack-years of smoking. Simulation studies show that the proposed methodology is sufficiently powered for realistic sample sizes and that it provides valid tests and effect size estimators in the presence of admixture and stratification.