Charge trapping and transport over chemical defects in polyethylene have significant impacts on its electrical and dielectric properties. However, the dynamics of this phenomenon and its underlying mechanisms remain unclear. To understand this fundamental aspect, we conducted a time-domain ab initio nonadiabatic molecular dynamics study of phonon-assisted holes dynamics in polyethylene over C═O and C-OH defect states. Our results suggest that the hole transfer and energy fluctuations substantially depend on temperature and local morphology. When the temperature decreases from 300 to 100 K, the hole transfer efficiency and the energy fluctuations are severely suppressed due to the weakened interactions between holes and phonons. Furthermore, amorphous polyethylene exhibits a severe suppression of the hole transfer process compared to crystalline polyethylene. An explanation for the influence of morphology on the hole transfer process can be found in the differences in the hole-phonon coupling and the electronic coupling between two chemical defect states in crystalline and amorphous polyethylene. Advancing the fundamental understanding of the dynamics of hole transfer over chemical effects in polymers is a key to improving their insulating properties for the next-generation high-voltage cables.