We report precision determinations of the beam-normal single spin asymmetries (A_{n}) in the elastic scattering of 0.95 and 2.18 GeV electrons off ^{12}C, ^{40}Ca, ^{48}Ca, and ^{208}Pb at very forward angles where the most detailed theoretical calculations have been performed. The first measurements of A_{n} for ^{40}Ca and ^{48}Ca are found to be similar to that of ^{12}C, consistent with expectations and thus demonstrating the validity of theoretical calculations for nuclei with Z≤20. We also report A_{n} for ^{208}Pb at two new momentum transfers (Q^{2}) extending the previous measurement. Our new data confirm the surprising result previously reported, with all three data points showing significant disagreement with the results from the Z≤20 nuclei. These data confirm our basic understanding of the underlying dynamics that govern A_{n} for nuclei containing ≲50 nucleons, but point to the need for further investigation to understand the unusual A_{n} behavior discovered for scattering off ^{208}Pb.