The behaviors of the light propagation in the phosphor play a vital role in determining the optical performance of the phosphor-converted light-emitting diodes (pc-LEDs). In this paper, we presented a general model based on the radiative transfer equation integrated with fluorescence (FRTE) to describe the overall light propagating properties in the phosphor layer in terms of light absorption, strong forward scattering, and fluorescence. The model was established by accounting for general boundary conditions including the LED Lambertian incidence, the diffuse reflection at the substrate/reflector, and the Fresnel reflection at the phosphor-air interface. The spectral element method (SEM) was extended to numerically solve FRTE. The radiant intensity at any location and direction of blue and yellow light was iteratively calculated, in which case the angular properties could be further evaluated. The model was validated by comparing the light extraction efficiency (LEE) and angular correlated color temperature (CCT) calculated by the presented model with the experimental results. Good agreements were achieved between model predictions and measurements with the corresponding maximum deviation of 4.9% and 3.7% for LEE and CCT, respectively. We also conducted a comparison between our model and the previous Kubelka-Munk (KM) theory. It has been revealed that the KM theory may overestimate the phosphor heating due to lacking of the blue light scattering effect.