Decreased intra-tumor heterogeneity (ITH) correlates with increased patient survival and immunotherapy response. However, even highly homogenous tumors may display variability in their aggressiveness, and how immunologic-factors impinge on their aggressiveness remains understudied. Here we studied the mechanisms responsible for the immune-escape of murine tumors with low ITH. We compared the temporal growth of homogeneous, genetically-similar single-cell clones that are rejected vs. those that are not-rejected after transplantation in-vivo using single-cell RNA sequencing and immunophenotyping. Non-rejected clones showed high infiltration of tumor-associated-macrophages (TAMs), lower T-cell infiltration, and increased T-cell exhaustion compared to rejected clones. Comparative analysis of rejection-associated gene expression programs, combined with in-vivo CRISPR knockout screens of candidate mediators, identified Mif (macrophage migration inhibitory factor) as a regulator of immune rejection. Mif knockout led to smaller tumors and reversed non-rejection-associated immune composition, particularly, leading to the reduction of immunosuppressive macrophage infiltration. Finally, we validated these results in melanoma patient data.