Responses to pictures of famous tennis and soccer athletes are slower when the responding effector is a hand or foot, respectively, indicating that visual recognition of individuals characterized by skilled motor behavior interferes with the motor reactivity of nonproficient observers. By contrast, directly viewing actions induces motor facilitation, suggesting that actions are mapped in the observers' motor system. Here, we used single-pulse Transcranial Magnetic Stimulation to determine 1) whether observing and recognizing the identity of famous tennis and soccer athletes selectively reduce the corticospinal excitability of arm and leg representations (categorization), 2) whether any athlete-related inhibition effect contrasts the facilitation associated with direct action observation (categorization + action), and 3) whether the classic action observation-related facilitation effect is found when viewing "in action" nonathlete models (action). In 3 experiments, we found that amplitude of motor evoked potentials (MEPs) recorded from leg and arm muscles gradually shifted from reduction to facilitation, moving from the categorization to the action observation tasks. Thus, semantic derivation of motor skills is reflected in limb-specific reduction of MEP amplitude, indicating that even abstract action knowledge is embodied in the motor system and that mapping others' actions on the basis of categorization or of their direct observation relies on competing functional mechanisms.