This study examined the role of TGF-beta1 in human keratinocyte malignancy. Two carcinoma-derived human oral keratinocyte cell lines, BICR 31 and H314, were selected on the basis of their known resistance to TGF-beta1-induced G(1) arrest, the presence of wild type TGF-beta cell surface receptors and normal Ras. Smad 4 protein was undetectable in both cell lines, but Smad 2 and Smad 3 were expressed at levels comparable with a fully TGF-beta responsive cell line, and treatment of the cells with TGF-beta1 resulted in the phosphorylation of Smad 2. Treatment with exogenous TGF-beta1 resulted in a failure to induce transcription from an artificial Smad-dependent promoter and a failure to down-regulate c-myc, but resulted in an up-regulation of AP-1 associated genes (Fra-1, JunB and fibronectin). Transient transfection of Smad 4 into BICR 31 restored TGF-beta1-induced growth inhibition and Smad-dependent transcriptional activation. Protracted treatment of cells with exogenous TGF-beta1 resulted in the attenuation of cell growth in vitro. To over-express TGF-beta1, both cell lines were transfected with latent TGF-beta1 cDNA; neutralization studies of conditioned media demonstrated that whilst the majority of the peptide was in the latent form, a small proportion was present as the active peptide. Cells that over-expressed endogenous TGF-beta1 grew more slowly in vitro compared to both the vector-only controls and cells that did not over-express the peptide. Orthotopic transplantation of cells that over-expressed endogenous TGF-beta1 to the floor of the mouth in athymic mice resulted in marked inhibition of primary tumor formation compared to controls. Expression of a dominant-negative TGF-beta type II receptor in cells that over-expressed endogenous TGF-beta1 resulted in enhanced cell growth in vitro and diminished the tumor suppressor effect of the ligand in vivo, indicating that the endogenous TGF-beta1 was acting in an autocrine capacity. The results demonstrate that over-expression of endogenous TGF-beta1 in human malignant oral keratinocytes leads to growth inhibition in vivo and tumor suppression in vitro by mechanisms that are independent of Smad 4 expression and TGF-beta1-induced G(1) arrest.