Vaccine effect, as measured in clinical trials, may not accurately reflect population-level impact. Furthermore, little is known about how sensitive apparent or real vaccine impacts are to factors such as the risk of re-infection or the mechanism of protection. We present a dynamic compartmental model to simulate vaccination for endemic infections. Several measures of effectiveness are calculated to compare the real and apparent impact of vaccination, and assess the effect of a range of infection and vaccine characteristics on these measures. Although broadly correlated, measures of real and apparent vaccine effectiveness can differ widely. Vaccine impact is markedly underestimated when primary infection provides partial natural immunity, when coverage is high and when post-vaccination infectiousness is reduced. Despite equivalent efficacy, 'all or nothing' vaccines are more effective than 'leaky' vaccines, particularly in settings with high risk of re-infection and transmissibility. Latent periods result in greater real impacts when risk of re-infection is high, but this effect diminishes if partial natural immunity is assumed. Assessments of population-level vaccine effects against endemic infections from clinical trials may be significantly biased, and vaccine and infection characteristics should be considered when modelling outcomes of vaccination programs, as their impact may be dramatic.