Caveolin-1 (Cav-1) expression in stromal carcinoma-associated fibroblasts (CAFs) has been associated with tumor progression and clinical outcome. This study was undertaken to assess its prognostic significance in invasive micropapillary carcinoma of the breast (IMPC), a tumor with abundant stromal CAFs and a high tendency for nodal metastasis and poor outcome. Cav-1 expression was studied by immunohistochemistry in a group of 86 cases of IMPC along with a control group of 105 cases of invasive ductal carcinoma, not otherwise specified (IDC-NOS). Our results indicate that absence of Cav-1 expression in CAFs of IMPC is more common than in IDC-NOS (57 %, 49/86 vs. 36 %, 38/105). The absence of expression was associated with larger tumor size and higher lymph node stage (P < 0.05) of IMPC. Univariate analysis suggested absence of Cav-1 in CAFs to be a candidate independent predictor of reduced progression-free survival (PFS) (HR = 3.945, 95 % CI = 1.717-9.063, P = 0.001), which was confirmed by multivariable analysis (P = 0.018). In patients with IMPC spreading to local lymph nodes, loss of stromal Cav-1 predicted a fourfold increase in risk for shortened PFS. In contrast, no significant difference of tumor epithelial Cav-1 expression was found between IMPC and IDC-NOS, and the expression of tumor Cav-1 was not significantly associated with the prognosis of patients with IMPC. Absence of Cav-1 expression in CAFs is a strong prognostic factor for IMPC patients, and it may further subgroup the patients with lymph node metastasis to guide clinical management.