Metal-organic frameworks (MOFs) have shown promising potential as proton-conducting materials due to their tunable structures and high porosity. In this study, two novel MOFs had been successfully synthesized, one containing sulfate groups (MOF-1; [Zn4(TIPE)2(SO4)4(H2O)]·5H2O) and the other containing sulfonate groups (MOF-2; [Zn2(TIPE)(5-sip)(NO3)0.66]·0.34NO3·17.5H2O) (TIPE = 1,1,2,2-tetrakis(4-(1H-imidazole-1-yl)phenyl)ethene, H35-sip = 5-sulfoisophthalicacid), and the effect of the two groups on the proton conductivity of Zn-based MOFs had been investigated and compared for the first time. The proton conductivity of these MOFs was systematically measured at different temperatures and humidity conditions. Remarkably, the results revealed significant differences in proton conductivity between the two sets of MOFs. At 90 °C and 98% RH, MOF-1 and MOF-2 achieved optimal proton conductivity of 4.48 × 10-3 and 5.69 × 10-2 S·cm-1, respectively. This was due to the structural differences arising from the presence of different functional groups, which subsequently affected the porosity and hydrophilicity, thereby influencing the proton conductivity. Overall, this comparative study revealed the influence of sulfate and sulfonate groups on the proton conductivity of Zn-based MOFs. This research provided a feasible idea for the development of advanced MOF materials with enhanced proton conductivity and opened up new possibilities for their application in proton devices.