A Comprehensive Comparison of Additional Benefit Assessment Methods Applied by Institute for Quality and Efficiency in Health Care and European Society for Medical Oncology for Time-to-Event Endpoints After Significant Phase III Trials-A Simulation Study

Objectives: After a successful Marketing Authorization Application for clinical trials with time-to-event endpoints, the degree of the added benefit from new treatments remains unknown and needs to be assessed. Unfortunately, until now no clear definition for added benefit determination of a treatment exists. Nevertheless, European authorities / societies have developed 2 "additional benefit assessment" methods, which have up to now not been compared: the European Society for Medical Oncology (ESMO) developed a dual rule considering relative and absolute benefit. The German Institute for Quality and Efficiency in Health Care (IQWiG) developed a method using upper 95% hazard ratio confidence interval.

Methods: We evaluate and compare both methods in an extensive simulation study including different censoring rates, failure time distributions, and treatment effects for sample size calculation. The methods' performance is assessed via Receiver Operating Characteristic curves, Spearman correlation, and percentage of achieved maximal scores.

Results: The results show that IQWiG's method has in many situations a lower maximal scoring proportion than ESMO's rule, that is, up to 28.5% versus 94.7%. Various failure time distributions lead to strongly changed maximal scoring percentages for ESMO. High positive correlation between the methods is present for moderate treatment effects.

Conclusions: IQWiG's method is usually more conservative than ESMO's. ESMO's rule tends to be more susceptible for various failure time distributions. Using the lower confidence interval limit seems to be a better solution resulting in a higher true-positive rate without increasing the false-positive rate. Thus, IQWiG's method might need to be adapted accordingly to achieve a better overall classification.