The role of post-operative radiotherapy (PORT) is still controversial for some cancer sites. In the absence of large randomized controlled trials, survival prediction models can help estimate the predicted benefit of PORT for specific settings. The purpose of this study was to compare the performance of two types of prediction models for estimating the benefit of PORT for 2 cancer sites. Using data from the Surveillance, Epidemiology, and End Results database, we constructed prediction models for gallbladder (GB) cancer and non-small cell lung cancer (NSMLC), using Cox proportional hazards and Random Survival Forests. We compared validation measures for discrimination and found that both the CPH and RSF models had comparable C-indices. For GB cancer, PORT was associated with improved survival for node positive patients, and for NSCLC, PORT was associated with a survival benefit for patients with N2 disease.