Cellulose-based dielectrics with attractive dielectric performance are promising candidates to develop eco-friendly electrostatic energy storage devices. Herein, all-cellulose composite films with superior dielectric constant were fabricated by manipulating the dissolution temperature of native cellulose, where we revealed the relationship among the hierarchical microstructure of the crystalline structure, the hydrogen bonding network, the relaxation behavior at a molecular level, and the dielectric performance of the cellulose film. The coexistence of cellulose I and cellulose II led to a weakened hydrogen bonding network and unstable C6 conformations. The increased mobility of cellulose chains in the cellulose I-amorphous interphase enhanced the dielectric relaxation strength of side groups and localized main chains. As a result, the as-prepared all-cellulose composite films exhibited a fascinating dielectric constant of as high as 13.9 at 1000 Hz. This work proposed here provides a significant step toward fundamentally understanding the dielectric relaxation of cellulose, thus developing high-performance and eco-friendly cellulose-based film capacitors.