A quaternary nonlinear-optical (NLO) zinc germanate, Rb2ZnGe2O6 (RZGO), was prepared from its stoichiometric melts, which belongs to the noncentrosymmetric space group C2221 (No. 20) and features a 3D framework formed by GeO4 tetrahedra, ZnO4 tetrahedra, and Rb+ cations filling into the void space. RZGO displays a good phase-matchable second-harmonic-generation (SHG) intensity similar to that of KH2PO4. The UV cutoff edge for RZGO was found to be approximately 0.37 μm with a broad optical band gap of 3.22 eV, indicating a large laser-induced damage threshold. Further characterizations suggested that this compound displays an excellent thermal stability (1014 °C) and melts congruently. Besides, theoretical analysis was also implemented to investigate the electronic structures and effective NLO coefficients as well as the origin of the observed SHG response for RZGO. The present study can enrich the diversity of acentric germanate structures and pave the way for the synthesis of new NLO compounds.