The 20S proteasome core particle (CP) is the proteolytically active key element of the ubiquitin proteasome system that directs the majority of intracellular protein degradation in eukaryotic cells. Over the past decade, the CP has emerged as an anticancer therapy target after approval of the first-in-class drug bortezomib (Velcade(®)) by the US Food and Drug Administration. However, bortezomib and all second-generation CP inhibitors that are currently explored in clinical phase studies react covalently and most often irreversibly with the proteolytic sites of the CP, hereby causing permanent CP blockage. Furthermore, reactive head groups result in unspecific binding to proteasomal active centers and in substantial enzymatic off-target activities that translate to severe side effects. Thus, reversible proteasome inhibitors might be a promising alternative, overcoming these drawbacks, but are challenging with respect to their urge for thorough enthalpic and entropic optimization. This review describes developments in the hitherto neglected field of reversible proteasome inhibitors focusing on insights gained from crystal structures, which provide valuable knowledge and strategies for future directions in drug development.