The role of orbital polarization for ultrafast charge transfer between an atomic adsorbate and a substrate is explored. Core hole clock spectroscopy with linearly polarized x-ray radiation allows to selectively excite adsorbate resonance states with defined spatial orientation relative to the substrate surface. For c(4 x 2)S/Ru(0001) the charge transfer times between the sulfur 2s(-1)3p*+1 antibonding resonance and the ruthenium substrate have been studied, with the 2s electron excited into the 3p perpendicular* state along the surface normal and the 3p parallel* state in the surface plane. The charge transfer times are determined as 0.18+/-0.07 and 0.84+/-0.23 fs, respectively. This variation is the direct consequence of the different adsorbate-substrate orbital overlap.