Influenza virus evolves to escape from immune system antibodies that bind to it. We used free energy calculations with Einstein crystals as reference states to calculate the difference of antibody binding free energy (ΔΔG) induced by amino acid substitution at each position in epitope B of the H3N2 influenza hemagglutinin, the key target for antibody. A substitution with positive ΔΔG value decreases the antibody binding constant. On average an uncharged to charged amino acid substitution generates the highest ΔΔG values. Also on average, substitutions between small amino acids generate ΔΔG values near to zero. The 21 sites in epitope B have varying expected free energy differences for a random substitution. Historical amino acid substitutions in epitope B for the A/Aichi/2/1968 strain of influenza A show that most fixed and temporarily circulating substitutions generate positive ΔΔG values. We propose that the observed pattern of H3N2 virus evolution is affected by the free energy landscape, the mapping from the free energy landscape to virus fitness landscape, and random genetic drift of the virus. Monte Carlo simulations of virus evolution are presented to support this view.