Spatially and temporally regulated membrane trafficking events incorporate membrane and cell wall materials into the pollen tube apex and are believed to underlie the rapid pollen tube growth. In plants, the molecular mechanisms and physiological functions of intra-Golgi transport and Golgi integrity maintenance remain largely unclear. The conserved oligomeric Golgi (COG) complex has been implicated in tethering of retrograde intra-Golgi vesicles in yeast and mammalian cells. Using genetic and cytologic approaches, we demonstrate that T-DNA insertions in Arabidopsis COG complex subunits, COG3 and COG8, cause an absolute, male-specific transmission defect that can be complemented by expression of COG3 and COG8 from the LAT52 pollen promoter, respectively. No obvious abnormalities in the microgametogenesis of the two mutants are observed, but in vitro and in vivo pollen tube growth are defective. COG3 or COG8 proteins fused to green fluorescent protein (GFP) label the Golgi apparatus. In pollen of both mutants, Golgi bodies exhibit altered morphology. Moreover, γ-COP and EMP12 proteins lose their tight association with the Golgi. These defects lead to the incorrect deposition of cell wall components and proteins during pollen tube growth. COG3 and COG8 interact directly with each other, and a structural model of the Arabidopsis COG complex is proposed. We believe that the COG complex helps to modulate Golgi morphology and vesicle trafficking homeostasis during pollen tube tip growth.