We report on a theoretical and experimental study of the energy transfer between an optical evanescent wave, propagating in vacuum along the planar boundary of a dielectric material, and a beam of sub-relativistic electrons. The evanescent wave is excited via total internal reflection in the dielectric by an infrared (λ = 2 μm) femtosecond laser pulse. By matching the electron propagation velocity to the phase velocity of the evanescent wave, energy modulation of the electron beam is achieved. A maximum energy gain of 800 eV is observed, corresponding to the absorption of more than 1000 photons by one electron. The maximum observed acceleration gradient is 19 ± 2 MeV/m. The striking advantage of this scheme is that a structuring of the acceleration element's surface is not required, enabling the use of materials with high laser damage thresholds that are difficult to nano-structure, such as SiC, Al2O3 or CaF2.