In order to understand the cellular events associated with cell death after the formation of topoisomerase II-DNA cleavable complexes, we compared the induction of endonucleolytic DNA fragmentation by etoposide and its more potent analog, teniposide (VM-26) in the human cell lines HT-29 and HL-60. A new filter-binding assay is described, which allows rapid quantification of nonprotein-linked DNA fragmentation involved in apoptosis. Both cell lines showed similar loss of colony formation ability following 30 min of treatment with various VM-26 concentrations even though the initial topoisomerase II-mediated DNA single-strand break frequency was higher in HL-60 cells. DNA repair studies following drug removal indicated that VM-26-induced DNA breaks reversed rapidly and completely in HT-29 cells, while in HL-60 cells, the initial lesions persisted at and above 5 microM VM-26. In both cell lines, topoisomerase II cleavage complexes, as measured by DNA-protein cross-links by alkaline elution, reversed rapidly and completely within 2-3 h. Secondary DNA fragmentation resembling chromatin endonucleolytic cleavage by apoptosis could be detected in HL-60 cells 3 h after VM-26 or etoposide treatment but not in HT-29 cells. Secondary DNA fragmentation was also induced in the human colon cancer cell lines COLO 320, which have c-myc amplification. Since HL-60 cells also have c-myc amplification and HT-29 do not, it is possible that c-myc overexpression may be involved in secondary DNA fragmentation. Finally, our results indicate heterogeneity of cell death mechanisms after exposure to topoisomerase II inhibitors among human cancer cell lines.