Proteins are translocated across membranes through a channel that is formed by the prokaryotic SecY or eukaryotic Sec61 complex. The crystal structure of the SecY channel from M. jannaschii revealed a plug domain that appears to seal the channel in its closed state. However, the role of the plug remains unclear, particularly because plug deletion mutants in S. cerevisiae are functional. Here, we demonstrate that plug deletion mutants in E. coli SecY are also functional and even efficiently translocate proteins with defective or missing signal sequences. The crystal structures of equivalent plug deletions in SecY of M. jannaschii show that, although the overall structures are maintained, new plugs are formed. These lack many interactions that normally stabilize the closed channel, explaining why the channels can open for proteins with signal-sequence mutations. Our data show that the plug domain is required to maintain a closed state of the channel and suggest a mechanism for channel gating.