Verrucomicrobia is a bacterial phylum that is commonly detected in soil, but little is known about the distribution and diversity of this phylum in the marine environment. To address this, we analyzed the marine microbial community composition in 506 samples from the International Census of Marine Microbes as well as 11 coastal samples taken from the California Current. These samples from both the water column and sediments covered a wide range of environmental conditions. Verrucomicrobia were present in 98% of the analyzed samples, and thus appeared nearly ubiquitous in the ocean. Based on the occurrence of amplified 16S ribosomal RNA sequences, Verrucomicrobia constituted on average 2% of the water column and 1.4% of the sediment bacterial communities. The diversity of Verrucomicrobia displayed a biogeography at multiple taxonomic levels and thus, specific lineages appeared to have clear habitat preference. We found that subdivision 1 and 4 generally dominated marine bacterial communities, whereas subdivision 2 was more frequent in low salinity waters. Within the subdivisions, Verrucomicrobia community composition were significantly different in the water column compared with sediment as well as within the water column along gradients of salinity, temperature, nitrate, depth and overall water column depth. Although we still know little about the ecophysiology of Verrucomicrobia lineages, the ubiquity of this phylum suggests that it may be important for the biogeochemical cycle of carbon in the ocean.