The COVID-19 pandemic has caused substantial worldwide disruptions in health, economy, and society, manifesting symptoms such as loss of smell (anosmia) and loss of taste (ageusia), that can result in prolonged sensory impairment. Establishing the host genetic etiology of anosmia and ageusia in COVID-19 will aid in the overall understanding of the sensorineural aspect of the disease and contribute to possible treatments or cures. By using human genome sequencing data from the University of Iowa (UI) COVID-19 cohort (N=187) and the National Institute of Health All of Us (AoU) Research Program COVID-19 cohort (N=947), we investigated the genetics of anosmia and/or ageusia by employing feature selection techniques to construct a novel variant and gene prioritization pipeline, utilizing machine learning methods for the classification of patients. Models were assessed using a permutation-based variable importance (PVI) strategy for final prioritization of candidate variants and genes. The highest held-out test set area under the receiver operating characteristic (AUROC) curve for models and datasets from the UI cohort was 0.735 and 0.798 for the variant and gene analysis respectively and for the AoU cohort was 0.687 for the variant analysis. Our analysis prioritized several novel and known candidate host genetic factors involved in immune response, neuronal signaling, and calcium signaling supporting previously proposed hypotheses for anosmia/ageusia in COVID-19.