The FYVE domain is a conserved protein motif characterized by its ability to bind with high affinity and specificity to phosphatidylinositol 3-phosphate (PI3P), a phosphoinositide highly enriched in early endosomes. The PI3P polar head group contacts specific amino acid residues that are conserved among FYVE domains. Despite full conservation of these residues, the ability of different FYVE domains to bind to endosomes in cells is highly variable. Here we show that the endosomal localization in intact cells absolutely requires structural features intrinsic to the FYVE domain in addition to the PI3P binding pocket. These features are involved in FYVE domain dimerization and in interaction with the membrane bilayer. These interactions, which are determined by non-conserved residues, are likely to be essential for the temporal and spatial control of protein associations at the membrane-cytosol interface within the endocytic pathway.