To avoid detection and targeting by the immune system, the plague-causing bacterium Yersinia pestis uses a type III secretion system to deliver a set of inhibitory proteins into the cytoplasm of immune cells. One of these proteins is an exceptionally active tyrosine phosphatase termed YopH, which paralyzes lymphocytes and macrophages by dephosphorylating critical tyrosine kinases and signal transduction molecules. Because Y. pestis strains lacking YopH are avirulent, we set out to develop small molecule inhibitors for YopH. We used a novel and cost-effective approach, in which leads from a chemical library screening were analyzed and computationally docked into the crystal structure of YopH. This resulted in the identification of a series of novel YopH inhibitors with nanomolar Ki values, as well as the structural basis for inhibition. Our inhibitors lack the polar phosphate-mimicking moiety of rationally designed tyrosine phosphatase inhibitors, and they readily entered live cells and rescued them from YopH-induced tyrosine dephosphorylation, signaling paralysis, and cell death. These inhibitors may become useful for treating the lethal infection by Y. pestis.