Benchmarking computational variant effect predictors by their ability to infer human traits

Daniel R Tabet; Da Kuang; Megan C Lancaster; Roujia Li; Karen Liu; Jochen Weile; Atina G Coté; Yingzhou Wu; Robert A Hegele; Dan M Roden; Frederick P Roth

doi:10.1186/s13059-024-03314-7

Benchmarking computational variant effect predictors by their ability to infer human traits

Genome Biol. 2024 Jul 1;25(1):172. doi: 10.1186/s13059-024-03314-7.

Authors

Daniel R Tabet^#^{1

2

3

4}, Da Kuang^#^{1

2

3

4}, Megan C Lancaster^#⁵, Roujia Li^{1

2

3

4}, Karen Liu^{1

2

3

4}, Jochen Weile^{1

2

3

4}, Atina G Coté^{1

2

3

4}, Yingzhou Wu^{1

2

3

4}, Robert A Hegele⁶, Dan M Roden^{5

7

8}, Frederick P Roth^{9

10

11

12

13}

Affiliations

¹ Donnelly Centre, University of Toronto, Toronto, ON, Canada.
² Department of Molecular Genetics, University of Toronto, Toronto, ON, Canada.
³ Department of Computer Science, University of Toronto, Toronto, ON, Canada.
⁴ Lunenfeld-Tanenbaum Research Institute, Sinai Health, Toronto, ON, Canada.
⁵ Division of Cardiovascular Medicine, Department of Medicine, Vanderbilt University Medical Center, Nashville, TN, USA.
⁶ Department of Medicine, Department of Biochemistry, Schulich School of Medicine and Dentistry, Robarts Research Institute, Western University, London, ON, Canada.
⁷ Department of Pharmacology, Vanderbilt University Medical Centre, Nashville, TN, USA.
⁸ Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN, USA.
⁹ Donnelly Centre, University of Toronto, Toronto, ON, Canada. [email protected].
¹⁰ Department of Molecular Genetics, University of Toronto, Toronto, ON, Canada. [email protected].
¹¹ Department of Computer Science, University of Toronto, Toronto, ON, Canada. [email protected].
¹² Lunenfeld-Tanenbaum Research Institute, Sinai Health, Toronto, ON, Canada. [email protected].
¹³ Department of Computational and Systems Biology, University of Pittsburgh School of Medicine, Pittsburgh, PA, USA. [email protected].

^# Contributed equally.

Abstract

Background: Computational variant effect predictors offer a scalable and increasingly reliable means of interpreting human genetic variation, but concerns of circularity and bias have limited previous methods for evaluating and comparing predictors. Population-level cohorts of genotyped and phenotyped participants that have not been used in predictor training can facilitate an unbiased benchmarking of available methods. Using a curated set of human gene-trait associations with a reported rare-variant burden association, we evaluate the correlations of 24 computational variant effect predictors with associated human traits in the UK Biobank and All of Us cohorts.

Results: AlphaMissense outperformed all other predictors in inferring human traits based on rare missense variants in UK Biobank and All of Us participants. The overall rankings of computational variant effect predictors in these two cohorts showed a significant positive correlation.

Conclusion: We describe a method to assess computational variant effect predictors that sidesteps the limitations of previous evaluations. This approach is generalizable to future predictors and could continue to inform predictor choice for personal and clinical genetics.

Keywords: All of Us; Benchmarking; Personal genomics; Rare missense variation; UK Biobank; Variant effect predictors.

Publication types

Research Support, N.I.H., Extramural
Research Support, Non-U.S. Gov't

MeSH terms

Benchmarking*
Computational Biology / methods
Genetic Variation*
Genotype
Humans
Phenotype

Abstract

Publication types

MeSH terms

Grants and funding