Increasing evidence suggests that the immune system affects prognosis of chronic myeloid leukemia (CML), but the detailed immunological composition of the leukemia bone marrow (BM) microenvironment is unknown. We aimed to characterize the immune landscape of the CML BM and predict the current treatment goal of tyrosine kinase inhibitor (TKI) therapy, molecular remission 4.0 (MR4.0). Using multiplex immunohistochemistry (mIHC) and automated image analysis, we studied BM tissues of CML patients (n = 56) and controls (n = 14) with a total of 30 immunophenotype markers essential in cancer immunology. CML patients' CD4+ and CD8+ T-cells expressed higher levels of putative exhaustion markers PD1, TIM3, and CTLA4 when compared to control. PD1 expression was higher in BM compared to paired peripheral blood (PB) samples, and decreased during TKI therapy. By combining clinical parameters and immune profiles, low CD4+ T-cell proportion, high proportion of PD1+TIM3-CD8+ T cells, and high PB neutrophil count were most predictive of lower MR4.0 likelihood. Low CD4+ T-cell proportion and high PB neutrophil counts predicted MR4.0 also in a validation cohort (n = 52) analyzed with flow cytometry. In summary, the CML BM is characterized by immune suppression and immune biomarkers predicted MR4.0, thus warranting further testing of immunomodulatory drugs in CML treatment.