Despite limited genomic diversity, SARS-CoV-2 has shown a wide range of clinical manifestations in different patient populations. The mechanisms behind these host differences are still unclear. Here, we examined host response gene expression across infection status, viral load, age, and sex among shotgun RNA-sequencing profiles of nasopharyngeal swabs from 430 individuals with PCR-confirmed SARS-CoV-2 and 54 negative controls. SARS-CoV-2 induced a strong antiviral response with upregulation of antiviral factors such as OAS1-3 and IFIT1-3 , and Th1 chemokines CXCL9/10/11 , as well as a reduction in transcription of ribosomal proteins. SARS-CoV-2 culture in human airway epithelial cultures replicated the in vivo antiviral host response. Patient-matched longitudinal specimens (mean elapsed time = 6.3 days) demonstrated reduction in interferon-induced transcription, recovery of transcription of ribosomal proteins, and initiation of wound healing and humoral immune responses. Expression of interferon-responsive genes, including ACE2 , increased as a function of viral load, while transcripts for B cell-specific proteins and neutrophil chemokines were elevated in patients with lower viral load. Older individuals had reduced expression of Th1 chemokines CXCL9/10/11 and their cognate receptor, CXCR3 , as well as CD8A and granzyme B, suggesting deficiencies in trafficking and/or function of cytotoxic T cells and natural killer (NK) cells. Relative to females, males had reduced B and NK cell-specific transcripts and an increase in inhibitors of NF-κB signaling, possibly inappropriately throttling antiviral responses. Collectively, our data demonstrate that host responses to SARS-CoV-2 are dependent on viral load and infection time course, with observed differences due to age and sex that may contribute to disease severity.