We report on the direct observation of key organic template-framework interactions leading to the formation of specific aluminophosphate structures. In particular, we show how MeAPO-34 formation was governed by an interaction between the divalent framework substituted metal ion and the template conformation, while for AlPO-5 the structure formation was determined by the template conformation alone. Understanding such interactions therefore appears to be important for the rationalization of microporous material formation.