Optical coherence elastography (OCE) maps the mechanical properties of tissue microstructure and has potential applications in both fundamental investigations of biomechanics and clinical medicine. We report the first analysis of contrast in OCE, including evaluation of the accuracy with which OCE images (elastograms) represent mechanical properties and the sensitivity of OCE to mechanical contrast within a sample. Using phase-sensitive compression OCE, we generate elastograms of tissue-mimicking phantoms with known mechanical properties and identify limitations on contrast imposed by sample mechanics and the imaging system, including signal-processing parameters. We also generate simulated elastograms using finite element models to perform mechanical analysis in the absence of imaging system noise. In both experiments and simulations, we illustrate artifacts that degrade elastogram accuracy, depending on sample geometry, elasticity contrast between features, and surface conditions. We experimentally demonstrate sensitivity to features with elasticity contrast as small as 1.1∶1 and calculate, based on our imaging system parameters, a theoretical maximum sensitivity to elasticity contrast of 1.002∶1. The results highlight the microstrain sensitivity of compression OCE, at a spatial resolution of tens of micrometers, suggesting its potential for the detection of minute changes in elasticity within heterogeneous tissue.