The pre-designable porous structures found in covalent organic frameworks (COFs) render them attractive as a molecular platform for addressing environmental issues such as removal of toxic heavy metal ions from water. However, a rational structural design of COFs in this aspect has not been explored. Here we report the rational design of stable COFs for Hg(II) removal through elaborate structural design and control over skeleton, pore size, and pore walls. The resulting framework is stable under strong acid and base conditions, possesses high surface area, has large mesopores, and contains dense sulfide functional termini on the pore walls. These structural features work together in removing Hg(II) from water and achieve a benchmark system that combines capacity, efficiency, effectivity, applicability, selectivity, and reusability. These results suggest that COFs offer a powerful platform for tailor-made structural design to cope with various types of pollution.