A novel compound barrier constructed with HCl modification zeolite and calcite was reported to control phosphorus and nitrogen release from sediments. Using short-term batch experiments and long-term sediment incubation experiments, the efficiency and mechanism of the compound barrier under anaerobic condition was investigated. For sediment incubation experiments, five kinds of active barrier conditions were applied respectively. The results showed that the calcite barrier reduced the phosphorus release from the sediment effectively, but not for the ammonia release from sediments. The compound barrier constructed with natural zeolite and calcite can reduce both the phosphorus and the ammonia release from sediments. In comparison to natural zeolite, the novel compound barrier constructed with HCl modification zeolite and calcite can control phosphorus released from sediments more effectively. Application of 40 g natural zeolite and 100 g calcite barrier resulted in 84% reduction of phosphorus flux from sediments for 80 days, but 91% reduction was obtained by compound barrier with 40 g HCl modification zeolite and 100 g calcite. In comparison with the natural zeolite, the duration of ammonia release control from sediments was shortened to two-thirds for the compound barrier with HCl modification zeolite. Reduction of Na+ and increment of H+ released from HCl modification zeolite were the possible reasons for the higher efficiency of the compound barrier to control the phosphorus release from the sediments.