Tangential flow filtration (TFF), which has been widely adopted to concentrate a diverse array of microbes from water, is a promising method of microbial separation or removal. However, it is essential to select an optimal membrane suitable for the specific filtration application. This study evaluated two different scales of TFF systems for concentrating and separating microbes (including bacteria and viruses) from contrasting marine waters. Among bacteria-size membranes, polyvinylidene difluoride (PVDF) membranes showed higher bacterial recovery, but lower viral permeation efficiencies than polyethersulfone (PES) membranes, regardless of environments and scales of TFF. Estuary samples showed significantly higher percentages of bacterial retention than nearshore and ocean samples. For virus-size membranes, a higher viral recovery and lower sorption was observed for regenerated cellulose membrane than PES membranes in the small-scale TFF. Similar viral recoveries were observed between PES membranes in the large-scale TFF, with higher viral concentrations being observed in estuary samples than in nearshore samples. Deep ocean samples showed the lowest recovery of viruses, which was consistent with observations of bacterial recovery. Synthetically, PVDF may be more suitable for the concentration of bacterial cells, while PES would be a better choice for the collection of viruses. When compared with the PES membrane, regenerated cellulose is better for viral concentration, while PES is recommended to obtain bacteria- and virus-free seawater.