We studied 54 ovarian smooth muscle tumors with an emphasis on histologic criteria for malignancy. Twenty-two leiomyomas were identified, including 7 typical, 11 cellular, 2 mitotically active, 1 with bizarre nuclei, and 1 myxoid. Follow-up ranging from 12 to 240 months (mean, 77.6 months) was available for 14 patients; all were alive with no evidence of disease. Of 26 leiomyosarcomas, including 2 myxoid leiomyosarcomas, most were readily diagnosed by the presence of at least two of the following: moderate or severe cytologic atypia, mitotic rate > or =10 mitotic figures per 10 high power fields, and tumor cell necrosis. Some cytologically atypical tumors demonstrated lesser mitotic activity of 5 to 9 mitotic figures per 10 high power fields, in the absence of tumor cell necrosis. Sixty percent of these were clinically malignant, supporting a diagnosis of leiomyosarcoma in such tumors. Follow-up was available for 21 patients. Seventy-one percent developed recurrent disease at a mean of 19 months, and 62% died of their disease at a mean of 24 months. Four tumors were deemed of uncertain malignant potential, and two that were stage II both recurred in the pelvis. One case of ovarian intravenous leiomyomatosis had a benign outcome at 42 months, as did one case of ovarian leiomyoma with leiomyomatosis peritonealis disseminata at 180 months. Overall, ovarian smooth muscle tumors encompass the same varied histologic spectrum as their uterine counterparts. The main tumors in the differential diagnosis are those in the fibroma/thecoma category, spindle cell carcinomas, and metastatic gastrointestinal stromal tumors.