We have developed omniBiomarker, a web-based application that uses knowledge from the NCI Cancer Gene Index to guide the selection of biologically relevant algorithms for identifying biomarkers. Biomarker identification from high-throughput genomic expression data is difficult because of data properties (i.e., small-sample size compared to large-feature size) as well as the large number of available feature selection algorithms. Thus, it is unclear which algorithm should be used for a particular dataset. These factors lead to instability in biomarker identification and affect the reproducibility of results. We introduce a method for computing the biological relevance of feature selection algorithms using an externally validated knowledge base of manually curated cancer biomarkers. Results suggest that knowledge-driven biomarker identification can improve microarray-based clinical prediction performance. omniBiomarker can be accessed at http://omnibiomarker.bme.gatech.edu/.