Porous 3D structures from mineralized collagen were fabricated applying a procedure in which collagen fibril reassembly and precipitation of nanocrystalline hydroxyapatite (HA) occur simultaneously. The resulting matrices were evaluated in vitro with respect to their suitability as scaffolds for bone tissue engineering. We found a high capacity of the material to bind serum proteins as well as to absorb Ca2+ ions, which could be advantageous to promote cell attachment, growth, and differentiation. Human bone marrow stromal cells (hBMSCs) were seeded onto the 3D scaffolds and cultivated for 4 weeks in the presence and absence of osteogenic supplements. We studied viability, proliferation, and osteogenic differentiation in terms of total lactate dehydrogenase (LDH) activity, DNA content, and alkaline phosphatase (ALP) activity. Furthermore, the expression for bone-related genes (ALP, bone sialo protein II (BSP II), and osteocalcin) was analyzed. In our investigation we found a 2.5-fold to 5-fold raise in DNA content and an increase of ALP activity for osteogenic induced hBMSC on collagen HA scaffolds. The expression of ALP and BSP II in these cells was also stimulated in the course of cultivation; however, we did not detect an upregulation of osteocalcin gene expression. These data suggest, that porous collagen HA scaffolds are suitable for the expansion and osteogenic differentiation of hBMSC and are therefore promising candidates for application as bone grafts.