Dentin and bone derive their mechanical properties from a complex arrangement of collagen type-I fibrils reinforced with nanocrystalline apatite mineral in extra- and intrafibrillar compartments. While mechanical properties have been determined for the bulk of the mineralized tissue, information on the mechanics of the individual fibril is limited. Here, atomic force microscopy was used on individual collagen fibrils to study structural and mechanical changes during acid etching. The characteristic 67 nm periodicity of gap zones was not observed on the mineralized fibril, but became apparent and increasingly pronounced with continuous demineralization. AFM-nanoindentation showed a decrease in modulus from 1.5 GPa to 50 MPa during acid etching of individual collagen fibrils and revealed that the modulus profile followed the axial periodicity. The nanomechanical data, Raman spectroscopy and SAXS support the hypothesis that intrafibrillar mineral etches at a substantially slower rate than the extrafibrillar mineral. These findings are relevant for understanding the biomechanics and design principles of calcified tissues derived from collagen matrices.