Much of the richness and variety of physics today are based on coupling phenomena where multiple interacting systems hybridize into new ones with completely distinct attributes. Recent development in building van der Waals (vdWs) heterostructures from different 2D materials provides exciting possibilities in realizing novel coupling phenomena in a designable manner. Here, with a graphene/hBN/graphene heterostructure, we report near-field infrared nano-imaging of plasmon-plasmon coupling in two vertically separated graphene layers. Emergent symmetric and anti-symmetric coupling modes are directly observed simultaneously. Coupling and decoupling processes are systematically investigated with experiment, simulation and theory. The reported interlayer plasmon-plasmon coupling could serve as an extra degree of freedom to control light propagation at the deep sub-wavelength scale with low loss and provide exciting opportunities for optical chip integration.