Many pathogens associated with chronic infections evolve so rapidly that strains found late in an infection have little in common with the initial strain. This raises questions at different levels of analysis because rapid within-host evolution affects the course of an infection, but it can also affect the possibility for natural selection to act at the between-host level. We present a nested approach that incorporates within-host evolutionary dynamics of a rapidly mutating virus (hepatitis C virus) targeted by a cellular cross-reactive immune response, into an epidemiological perspective. The viral trait we follow is the replication rate of the strain initiating the infection. We find that, even for rapidly evolving viruses, the replication rate of the initial strain has a strong effect on the fitness of an infection. Moreover, infections caused by slowly replicating viruses have the highest infection fitness (i.e., lead to more secondary infections), but strains with higher replication rates tend to dominate within a host in the long-term. We also study the effect of cross-reactive immunity and viral mutation rate on infection life history traits. For instance, because of the stochastic nature of our approach, we can identify factors affecting the outcome of the infection (acute or chronic infections). Finally, we show that anti-viral treatments modify the value of the optimal initial replication rate and that the timing of the treatment administration can have public health consequences due to within-host evolution. Our results support the idea that natural selection can act on the replication rate of rapidly evolving viruses at the between-host level. It also provides a mechanistic description of within-host constraints, such as cross-reactive immunity, and shows how these constraints affect the infection fitness. This model raises questions that can be tested experimentally and underlines the necessity to consider the evolution of quantitative traits to understand the outcome and the fitness of an infection.