Unconventional superconductivity often emerges at the border of long-range magnetic orders. Understanding the low-energy charge dynamics may provide crucial information on the formation of superconductivity. Here we report the unpolarized/polarized optical conductivity study of high quality MnP single crystals at ambient pressure. Our data reveal two types of charge carriers with very different lifetimes. In combination with the first-principles calculations, we show that the short-lifetime carriers have flat Fermi sheets which become gapped in the helimagnetic phase, causing a dramatic change in the low-frequency optical spectra, while the long-lifetime carriers are anisotropic three-dimensional like which are little affected by the magnetic transitions and provide major contributions to the transport properties. This orbital-dependent charge dynamics originates from the special crystal structure of MnP and may have an influence on the unconventional superconductivity and its interplay with helimagnetism at high pressures.