Two phosphates, KZnPO4 and NH4ZnPO4, were successfully synthesized using a solid-state reaction method and characterized via single-crystal X-ray diffraction. Both of their structures feature a three-way skeleton composed of isolated PO4 and ZnO4 tetrahedra, with K+/NH4+ cations occupying the spaces within the frameworks to balance charges. These compounds are isostructural and crystallize in the noncentrosymmetric P63 space group. Their structures were compared, revealing that non-centrosymmetric (NCS) KZnPO4 and NH4ZnPO4 exhibit second harmonic generation (SHG) responses of 0.4 × KDP and 1 × KDP, respectively. Furthermore, by combining the electronic structure and SHG density calculations, we examined the effects of ZnO4 tetrahedra on the SHG response of KZnPO4 and NH4ZnPO4.