SapM is a secreted virulence factor from Mycobacterium tuberculosis critical for pathogen survival and persistence inside the host. Its full potential as a target for tuberculosis treatment has not yet been exploited because of the lack of potent inhibitors available. By screening over 1500 small molecules, we have identified new potent and selective inhibitors of SapM with an uncompetitive mechanism of inhibition. The best inhibitors share a trihydroxy-benzene moiety essential for activity. Importantly, the inhibitors significantly reduce mycobacterial burden in infected human macrophages at 1 µM, and they are selective with respect to other mycobacterial and human phosphatases. The best inhibitor also reduces intracellular burden of Francisella tularensis, which secretes the virulence factor AcpA, a homologue of SapM, with the same mechanism of catalysis and inhibition. Our findings demonstrate that inhibition of SapM with small molecule inhibitors is efficient in reducing intracellular mycobacterial survival in host macrophages and confirm SapM as a potential therapeutic target. These initial compounds have favourable physico-chemical properties and provide a basis for exploration towards the development of new tuberculosis treatments. The efficacy of a SapM inhibitor in reducing Francisella tularensis intracellular burden suggests the potential for developing broad-spectrum antivirulence agents to treat microbial infections.