The conversion of latent transforming growth factor beta (LTGF-beta) to the active species, transforming growth factor beta (TGF-beta), has been characterized in heterotypic cultures of bovine aortic endothelial (BAE) cells and bovine smooth muscle cells (SMCs). The formation of TGF-beta in co-cultures of BAE cells and SMCs was documented by a specific radioreceptor competition assay, while medium from homotypic cultures of BAE cells or SMCs contained no active TGF-beta as determined by this assay. The concentration of TGF-beta in the conditioned medium of heterotypic co-cultures was estimated to be 400-1,200 pg/ml using the inhibition of BAE cell migration as an assay. Northern blotting of poly A+ RNA extracted from both homotypic and heterotypic cultures of BAE cells and SMCs revealed that BAE cells produced both TGF-beta 1 and TGF-beta 2, while SMCs produced primarily TGF-beta 1. No change in the expression of these two forms of TGF-beta was apparent after 24 h in heterotypic cultures. Time course studies on the appearance of TGF-beta indicated that most of the active TGF-beta was generated within the first 12 h after the establishment of co-cultures. The generation of TGF-beta in co-cultures stimulated the production of the protease inhibitor plasminogen activator inhibitor-1 (PAI-1). The inclusion of neutralizing antibodies to TGF-beta in the co-culture medium blocked the observed increase in PAI-1 levels. The increased expression of PAI-1 subsequent to TGF-beta formation blocked the activation of the protease required for conversion of LTGF-beta to TGF-beta as the inclusion of neutralizing antibodies to PAI-1 in the co-culture medium resulted in prolonged production of TGF-beta. This effect was lost upon removal of the PAI-1 antibodies. Thus, the activation of LTGF-beta appears to be a self-regulating system.