Predicting lack of clinical improvement following varicose vein ablation using machine learning

Objective: Varicose vein ablation is generally indicated in patients with active/healed venous ulcers. However, patient selection for intervention in individuals without venous ulcers is less clear. Tools that predict lack of clinical improvement (LCI) following vein ablation may help guide clinical decision-making but remain limited. We developed machine learning (ML) algorithms that predict 1-year LCI following varicose vein ablation.

Methods: The Vascular Quality Initiative (VQI) database was used to identify patients who underwent endovenous or surgical varicose vein treatment for Clinical-Etiological-Anatomical-Pathophysiological (CEAP) C2-C4 disease between 2014-2024. We identified 226 predictive features (111 pre-operative [demographic/clinical], 100 intra-operative [procedural], and 15 post-operative [immediate post-operative course/complications]). The primary outcome was 1-year LCI, defined as a pre-operative venous clinical severity score (VCSS) minus post-operative VCSS ≤ 0, indicating no clinical improvement following vein ablation. The data was divided into training (70%) and test (30%) sets. Six ML models were trained using pre-operative features with 10-fold cross-validation (Extreme Gradient Boosting [XGBoost], random forest, Naïve Bayes classifier, support vector machine, artificial neural network, and logistic regression). The primary model evaluation metric was area under the receiver operating characteristic curve (AUROC). The algorithm with the best performance was further trained using intra- and post-operative features. The focus was on pre-operative features, while intra- and post-operative features were of secondary importance, because pre-operative predictions offer the most potential to mitigate risk, such as deciding whether to proceed with intervention. Model calibration was assessed using calibration plots, while the accuracy of probabilistic predictions was evaluated with Brier scores. Performance was evaluated across subgroups based on age, sex, race, ethnicity, rurality, median Area Deprivation Index, prior ipsilateral varicose vein ablation, location of primary vein treated, and treatment type.

Results: Overall, 33,924 patients underwent varicose vein treatment (30,602 [90.2%] endovenous and 3,322 [9.8%] surgical) during the study period and 5,619 (16.6%) experienced 1-year LCI. Patients who developed the primary outcome were older, more likely to be socioeconomically disadvantaged, and less likely to routinely use compression therapy. They also had less severe disease as characterized by lower pre-operative VCSS, VVSymQ scores, and CEAP classifications. The best pre-operative prediction model was XGBoost, achieving an AUROC (95% CI) of 0.94 (0.93-0.95). In comparison, logistic regression had an AUROC (95% CI) of 0.71 (0.70-0.73). The XGBoost model had marginally improved performance at the intra- and post-operative stages, both achieving an AUROC (95% CI) of 0.97 (0.96-0.98). Calibration plots showed good agreement between predicted and observed event probabilities with Brier scores of 0.12 (pre-operative), 0.11 (intra-operative), and 0.10 (post-operative). Of the top 10 predictors, 7 were pre-operative features including VCSS, VVSymQ score, CEAP classification, prior varicose vein ablation, thrombus in the greater saphenous vein, and reflux in the deep veins. Model performance remained robust across all subgroups.

Conclusions: We developed ML models that can accurately predict outcomes following endovenous and surgical varicose vein treatment for CEAP C2-C4 disease, performing better than logistic regression. These algorithms have potential for important utility in guiding patient counseling and peri-operative risk mitigation strategies to prevent LCI following varicose vein ablation.