Thermodynamic integration (TI) was combined with (adaptive) umbrella sampling to improve the convergence of alchemical free energy simulations in which multiple conformational substates are present. The approach, which we refer to as non-Boltzmann TI (NBTI), was tested by computing the free energy differences between three five-atomic model systems, as well as the free energy difference of solvation between leucine and asparagine. In both cases regular TI failed to give converged results, whereas the NBTI results were free from hysteresis and had standard deviations well below +/-0.7 kcal/mole. We also present theoretical considerations that make it possible to compute free energy differences between simple molecules, such as the five-atomic model systems, by numerical integration of the partition functions at the respective end points.