Ce(3+)/Eu(2+), Tb(3+) and Mn(2+) co-doping in single-phase hosts is a common strategy to achieve white-light phosphors via energy transfer, which provides a high color rendering index (CRI) value and good color stability. However, not all hosts are suitable for white-light phosphors due to inefficient energy transfer. In this study, the site-sensitive energy transfer from different crystallographic sites of Ce(3+) to Tb(3+)/Mn(2+) in Ca3Al2O6 has been investigated in detail. The energy transfer from purplish-blue Ce(3+) to Tb(3+) is an electric dipole-dipole mode, and the calculated critical distance (Rc) suggests the existence of purplish-blue Ce(3+)-Tb(3+) clusters. No energy transfer is observed from purplish-blue Ce(3+) to Mn(2+). In co-doped phosphors based on greenish-blue Ce(3+), however, the radiative mode dominates the energy transfer from Ce(3+) to Tb(3+), and an electric dipole-quadrupole interaction is responsible for the energy transfer from Ce(3+) to Mn(2+). A detailed discussion on the site-sensitive energy transfer modes might provide a new aspect to discuss and understand the possibilities and mechanisms of energy transfer, according to certain crystallographic sites in a complex host with different cation sites, as well as provide a possible approach in searching for single-phase white-light-emitting phosphors.