Intimal hyperplasia is characterized by smooth muscle cell (SMC) dedifferentiation from a contractile to a synthetic phenotype prior to migration and proliferation. Regulatory mechanisms controlling SMC phenotype are not well known. This study examined the effect of endothelial cells (ECs) on SMC morphology in coculture. Subcultured bovine ECs and SMCs were plated on opposite sides of a 13 microns thick, semipermeable membrane (0.45 micron pores, Cyclopore) to allow potential humoral and cellular cross-membrane communication. SMCs were studied (5 wells/group) in coculture opposite confluent ECs (EC/SMC) and alone (SMC controls). After 4 days of culture in Dulbecco's modified Eagle medium/2.5% calf serum, SMCs were harvested. The ratio of protein/DNA was measured as an index of SMC hypertrophy (synthetic SMC phenotype). SMCs were examined with light and scanning electron microscopy to evaluate cell surface area, cellular morphology, and macroscopic growth characteristics. Flow cytometry was used to determine the cellular RNA/DNA ratio. SMC control cultures had a significantly greater protein-to-DNA content than SMCs cocultured with ECs (175 +/- 9 vs. 115 +/- 7 micrograms protein/micrograms DNA; p < 0.001). SMC control cultures also had 6.5 times greater cell surface area (5.8 +/- 0.3 x 10(3) microns2) than cocultured SMCs (0.9 +/- 0.1; p < 0.001). In SMC control cultures, SMC hypertrophy and rapid "hill and valley" formation were observed. In contrast, SMCs from the EC/SMC group exhibited a more spindle-shaped, contractile-appearing phenotype with more uniform, evenly distributed cells and no hill and valley formation. SMC control cultures also had a higher RNA/DNA ratio. Thus the presence of confluent ECs substantially altered the morphology and growth characteristic normally observed for SMCs in vitro. This coculture system provides a model to further study EC-SMC interaction, which could have important in vivo consequences.