Reinfections with respiratory viruses such as influenza viruses and coronaviruses are thought to be driven by ongoing antigenic immune escape in the viral population. However, this does not explain why antigenic variation is frequently observed in these viruses relative to viruses such as measles that undergo systemic replication. Here, we suggest that the rapid rate of waning immunity in the respiratory tract is the key driver of antigenic evolution in respiratory viruses. Waning immunity results in hosts with immunity levels that protect against homologous reinfection but are insufficient to protect against infection with a heterologous, antigenically different strain. As such, when partially immune hosts are present at a high enough density, an immune escape variant can invade the viral population even though that variant cannot infect fully immune hosts. Invasion can occur even when the variant's immune escape mutation incurs a fitness cost, and we expect the expanding mutant population will evolve compensatory mutations that mitigate this cost. Thus the mutant lineage should replace the wild-type, and as immunity to it builds, the process will repeat. Our model provides a new explanation for the pattern of successive emergence and replacement of antigenic variants that has been observed in many respiratory viruses. We discuss our model relative to others for understanding the drivers of antigenic evolution in these and other respiratory viruses.