Vietnam exhibits great cultural and linguistic diversity, yet the genetic history of Vietnamese populations remains poorly understood. Previous studies focused mostly on the majority Kinh group, and thus the genetic diversity of the many other groups has not yet been investigated. Here we analyze complete mtDNA genome sequences and ~2.3 Mb sequences of the male-specific portion of the Y chromosome from the Kinh and 16 minority populations, encompassing all five language families present in Vietnam. We find highly variable levels of diversity within and between groups that do not correlate with either geography or language family. In particular, the Mang and Sila have undergone recent, independent bottlenecks, while the majority group, Kinh, exhibits low levels of differentiation with other groups. The two Austronesian-speaking groups, Giarai and Ede, show a potential impact of matrilocality on their patterns of variation. Overall, we find that isolation, coupled with limited contact involving some groups, has been the major factor influencing the genetic structure of Vietnamese populations, and that there is substantial genetic diversity that is not represented by the Kinh.