We used functional Magnetic Resonance Imaging to explore the neural correlates of semantic relevance in 12 healthy participants performing a picture-naming task. In addition, we tested the hypothesis that category effects typically found in functional imaging can be partly explained in terms of different semantic relevance for animals and artefacts. We report that semantic relevance modulates neuronal responses in the medial fusiform gyrus bilaterally. As predicted, category effects in this region are strongly modulated by the semantic relevance of the items. Specifically, the effect of artefacts > animals is greatly reduced when the two categories are matched for semantic relevance. Thus, the present study demonstrates that neuronal responses during concept retrieval are modulated by the semantic relevance of the features. It also suggests that increased activation in the medial fusiform gyrus typically found for artefacts > animals can be explained by different semantic relevance for animal and artefact items.