Understanding interplay of breast cancer and microenvironment is critical. Here, we identified two transcriptomic subtypes and five immune infiltration patterns from RNA-seq and multiplex immunohistochemistry from 21 ER+/HER2- invasive lobular breast cancers. The proliferative subtype associated with increased immune infiltration especially by immunosuppressive regulatory T-cells and macrophages. We also defined a TAM-Low signature, which associated with lower infiltration of proliferative, pro-inflammatory TAM, and improved outcome in patients with ER+ tumors.