Soluble guanylate cyclase (sGC) plays a central role in the cardiovascular system and is a drug target for the treatment of pulmonary hypertension. While the three-dimensional structure of sGC is unknown, studies suggest that binding of the regulatory domain to the catalytic domain maintains sGC in an autoinhibited basal state. The activation signal, binding of NO to heme, is thought to be transmitted via the regulatory and dimerization domains to the cyclase domain and unleashes the full catalytic potential of sGC. Consequently, isolated catalytic domains should show catalytic turnover comparable to that of activated sGC. Using X-ray crystallography, activity measurements, and native mass spectrometry, we show unambiguously that human isolated catalytic domains are much less active than basal sGC, while still forming heterodimers. We identified key structural elements regulating the dimer interface and propose a novel role for residues located in an interfacial flap and a hydrogen bond network as key modulators of the orientation of the catalytic subunits. We demonstrate that even in the absence of the regulatory domain, additional sGC domains are required to guide the appropriate conformation of the catalytic subunits associated with high activity. Our data support a novel regulatory mechanism whereby sGC activity is tuned by distinct domain interactions that either promote or inhibit catalytic activity. These results further our understanding of heterodimerization and activation of sGC and open additional drug discovery routes for targeting the NO-sGC-cGMP pathway via the design of small molecules that promote a productive conformation of the catalytic subunits or disrupt inhibitory domain interactions.