The estrogen receptor is capable of binding a diverse set of ligands that are broadly categorized as agonists or antagonists, depending on their abilities to induce or interfere with transcriptional responsiveness. We show, using a fusion protein assay for ligand-binding which does not rely on transcriptional responsiveness, that agonists and antagonists differently position the C-terminus of the ligand-binding domain (helix 12) and the F domain. Upon antagonist binding, the F domain interferes with the fusion protein activity. Mutational disruption of helix 12 alters the position of the F domain, imposing interference after agonist or antagonist binding. Genetically selected inversion mutations where only agonists, but not antagonists, induce interference are similarly reliant on helix 12 and F domain positioning. Our results demonstrate that agonists and antagonists differently position helix 12 and implicate the F domain in mechanisms of antagonist action.