Phyllodes tumor (PT) of the breast is a rare but clinically important fibroepithelial tumor with potential risks of recurrence and metastasis. Recent studies identified recurrent TERT promoter mutations in PTs. However, the clinical significance of this alteration has not been fully examined. Two hundred and seven PTs from two intuitions were included. All cases were subjected to immunohistochemical analysis for TERT expression. Analysis of TERT promoter mutations was further performed by Sanger sequencing targeting the hotspot mutation region on cases from one of the involved institutions. The expression of TERT was correlated with clinicopathologic features, mutation status and recurrence. There was an association of TERT expression and its promoter mutation. Both stromal TERT expression and its promoter mutation correlated with PT grading and older patient age. Recurrence free survival (RFS) of PT patients with high stromal TERT expression was shorter if the excision margin was positive. Our findings suggested a possible pathogenic role of TERT alteration in PT malignancy. Currently there is no consensus for re-excision for PT patients with positive surgical margin, particularly for low grade cases. Stromal TERT expression could be potentially useful to guide management patients with benign PTs.