Tissue engineering approaches to stimulate bone formation currently combine bioactive scaffolds with osteocompetent human mesenchymal stem cells (hMSC). Moreover, osteogenic and angiogenic factors are required to promote differentiation and survival of hMSC through improved vascularization through the damaged extracellular matrix (ECM). Glycosaminoglycans (GAGs) are ECM compounds acting as modulators of heparin-binding protein activities during bone development and regenerative processes. GAG mimetics have been proposed as ECM stabilizers and were previously described for their positive effects on bone formation and angiogenesis after local treatment. Here, we developed a strategy associating the GAG mimetic [OTR4120] with bone substitutes to optimize stem cell-based therapeutic products. We showed that [OTR4120] was able to potentiate proliferation, migration, and osteogenic differentiation of hMSC in vitro. Its link to tricalcium phosphate/hydroxyapatite scaffolds improved their colonization by hMSC. Surprisingly, when these combinations were tested in an ectopic model of bone formation in immunodeficient mice, the GAG mimetics inhibit bone formation induced by hMSC and promoted an osteoclastic activity. Moreover, the inflammatory response was modulated, and the peri-implant vascularization stimulated. All together, these findings further support the ability of GAG mimetics to organize the local ECM to coordinate the host response toward the implanted biomaterial, and to inhibit the abnormal bone formation process on a subcutaneous ectopic site.