Sirtuins are NAD+-dependent protein deacetylases regulating metabolism, stress responses, and aging processes. Mammalia possess seven Sirtuin isoforms, Sirt1-7, which differ in their subcellular localization and in the substrate proteins they deacetylate. The physiological roles of Sirtuins and their potential use as therapeutic targets for metabolic and aging-related diseases have spurred interest in the development of small-molecule Sirtuin modulators. Here, we describe an approach exploiting the structures available for four human Sirtuins for the development of isoform-specific inhibitors. Virtual docking of a compound library into the peptide binding pockets of crystal structures of Sirt2, 3, 5 and 6 yielded compounds potentially discriminating between these isoforms. Further characterization in activity assays revealed several inhibitory compounds with little isoform specificity, but also two compounds with micromolar potency and high specificity for Sirt2. Structure comparison and the predicted, shared binding mode of the Sirt2-specific compounds indicate a pocket extending from the peptide-binding groove as target side enabling isoform specificity. Our family-wide structure-based approach thus identified potent, Sirt2-specific inhibitors as well as lead structures and a target site for the development of compounds specific for other Sirtuin isoform, constituting an important step toward the identification of a complete panel of isoform-specific Sirtuin inhibitors.