Continuing efforts to quantify the influence of aerosol light absorption upon global heat budgets rely on high-quality measurements of aerosol optical properties. Of the available methods, photoacoustic spectroscopy stands out as a sensitive method for measurements of aerosol absorption with minimal sample modification. Theoretical treatments of photoacoustic aerosol detection have predicted size-dependent damping of the photoacoustic signal as a result of particle thermal inertia. We provide experimental confirmation of this prediction using a single-particle photoacoustic spectrometer, which allows us to measure photoacoustic signals with high sensitivity and size-specificity. Both the magnitude and phase of the photoacoustic response follow the linearized description of the heat flux. The quantification of this effect provides a basis for future, system-specific case studies.