Alkaloids with allelopathic activity are not as well-known as other allelochemicals. Our study revealed that total alkaloids from seeds of the medicinal plant Peganum harmala L. possessed significant growth inhibitory effect on four treated plants, with dicot plants (lettuce and amaranth) being more sensitive than the tested monocot plants (wheat and ryegrass). Further investigation led to the isolation of harmaline and harmine as the main active ingredients in the total alkaloids of P. harmala seeds. Harmaline exerted potent inhibitory effects on seedling growth of treated plants, especially dicots, inhibiting root elongation of lettuce and amaranth by 31% and 47% at a very low concentration (5 µg/mL), whereas harmine exhibited much weaker non-selective inhibitory effect on the plants. Considering the high yield and poor utilization of P. harmala in China, we anticipate that this plant could be exploited as an alternative weed management tool in the future.