The rates of fine structure relaxation in PbS, PbSe, and PbTe nanocrystals were measured on a femtosecond time scale as a function of temperature with no applied magnetic field by cross-polarized transient grating spectroscopy (CPTG) and circularly polarized pump-probe spectroscopy. The relaxation rates among exciton fine structure states follow trends with nanocrystal composition and size that are consistent with the expected influence of material dependent spin-orbit coupling, confinement enhanced electron-hole exchange interaction, and splitting between L valleys that are degenerate in the bulk. The size dependence of the fine structure relaxation rate is considerably different from what is observed for small CdSe nanocrystals, which appears to result from the unique material properties of the highly confined lead chalcogenide quantum dots. Modeling and qualitative considerations lead to conclusions about the fine structure of the lowest exciton absorption band, which has a potentially significant bearing on photophysical processes that make these materials attractive for practical purposes.