While the phase diagrams of the one- and multiorbital Hubbard model have been well studied, the physics of real Mott insulators is often much richer, material dependent, and poorly understood. In the prototype Mott insulator V_{2}O_{3}, chemical pressure was initially believed to explain why the paramagnetic-metal to antiferromagnetic-insulator transition temperature is lowered by Ti doping while Cr doping strengthens correlations, eventually rendering the high-temperature phase paramagnetic insulating. However, this scenario has been recently shown both experimentally and theoretically to be untenable. Based on full structural optimization, we demonstrate via the charge self-consistent combination of density functional theory and dynamical mean-field theory that changes in the V_{2}O_{3} phase diagram are driven by defect-induced local symmetry breakings resulting from dramatically different couplings of Cr and Ti dopants to the host system. This finding emphasizes the high sensitivity of the Mott metal-insulator transition to the local environment and the importance of accurately accounting for the one-electron Hamiltonian, since correlations crucially respond to it.