Histone deacetylases (HDACs) regulate chromatin status and gene expression, and their inhibition is of significant therapeutic interest. To date, no biological substrate for class IIa HDACs has been identified, and only low activity on acetylated lysines has been demonstrated. Here, we describe inhibitor-bound and inhibitor-free structures of the histone deacetylase-4 catalytic domain (HDAC4cd) and of an HDAC4cd active site mutant with enhanced enzymatic activity toward acetylated lysines. The structures presented, coupled with activity data, provide the molecular basis for the intrinsically low enzymatic activity of class IIa HDACs toward acetylated lysines and reveal active site features that may guide the design of class-specific inhibitors. In addition, these structures reveal a conformationally flexible structural zinc-binding domain conserved in all class IIa enzymes. Importantly, either the mutation of residues coordinating the structural zinc ion or the binding of a class IIa selective inhibitor prevented the association of HDAC4 with the N-CoR.HDAC3 repressor complex. Together, these data suggest a key role of the structural zinc-binding domain in the regulation of class IIa HDAC functions.