Size dependence of spin dynamics in colloidal CdSe quantum dots (QDs) are investigated with circularly polarized pump-probe transmission spectroscopy at room temperature. The excitation energy is tuned to resonance with the lowest exciton (1S(h)1S(e)) energy of the CdSe QDs. The exciton spin dynamics of CdSe QD with the diameter of 5.2 nm shows monoexponential decay with a typical time constant of about 1-3 ps depending on the excitation energy. For the cases of CdSe QDs with smaller size (with the diameter of 4.0 and 2.4 nm), the exciton spin relaxation shows biexponential decay, a fast component with time constant of several ps and a slow one with time constant of hundreds of ps to nanosecond time scale. The fast spin relaxation arises from the bright-dark transition, i.e., J = ±1 ↔ -/+2 transition. This process is dominated by the hole spin flips, while the electron spin conserves. The slow spin relaxation is attributed to the intralevel exciton transitions (J = ±1 ↔ -/+1 transition), which is relevant to the electron spin flip. Our results indicate that the exciton spin relaxation pathways in CdSe QD are controllable by monitoring the particle size, and polarized pump-probe spectroscopy is proved to be a sensitive method to probe the exciton transition among the fine structures.