We study the origin of the scaling behavior in frictionless granular media above the jamming transition by analyzing their linear response. The response to local forcing is non-self-averaging and fluctuates over a length scale that diverges at the jamming transition. The response to global forcing becomes increasingly nonaffine near the jamming transition. This is due to the proximity of floppy modes, the influence of which we characterize by the local linear response. We show that the local response also governs the anomalous scaling of elastic constants and contact number.