The photothermal heterodyne imaging method is used to study for the first time the absorption spectra of individual gold nanoparticles with diameters down to 5 nm. Intrinsic size effects that result in a broadening of the surface plasmon resonance are unambiguously observed. Dispersions in the peak energies and homogeneous widths of the single-particle resonances are revealed. The experimental results are analyzed within the frame of Mie theory.