The polymer center-of-mass tracer diffusion coefficient in athermal polymer nanocomposites (PNCs) composed of polystyrene and phenyl-capped, spherical silica nanoparticles was measured over a range of temperatures and nanoparticle concentrations using elastic recoil detection. The polymer tracer diffusion coefficient in the PNC relative to the bulk decreases with increasing nanoparticle concentration and is unexpectedly more strongly reduced at higher temperatures. This unusual temperature dependence of polymer diffusion in PNCs cannot be explained by the reptation model or a modified version incorporating an effective tube diameter. Instead we show that our results are consistent with a mechanism based on nanoparticle-imposed configurational entropy barriers.