Janus kinase (JAK) inhibitors are being developed for the treatment of rheumatoid arthritis, psoriasis, myeloproliferative neoplasms, and leukemias. Most of these drugs target the ATP-binding pocket and stabilize the active conformation of the JAK kinases. This type I binding mode can lead to an increase in JAK activation loop phosphorylation, despite blockade of kinase function. Here we report that stabilizing the inactive state via type II inhibition acts in the opposite manner, leading to a loss of activation loop phosphorylation. We used X-ray crystallography to corroborate the binding mode and report for the first time the crystal structure of the JAK2 kinase domain in an inactive conformation. Importantly, JAK inhibitor-induced activation loop phosphorylation requires receptor interaction, as well as intact kinase and pseudokinase domains. Hence, depending on the respective conformation stabilized by a JAK inhibitor, hyperphosphorylation of the activation loop may or may not be elicited.