Heterogeneity is a hallmark of cancer whether one considers the genotype of cancerous cells, the composition of their microenvironment, the distribution of blood and lymphatic microvasculature, or the spatial distribution of the desmoplastic reaction. It is logical to expect that this heterogeneity in tumor microenvironment will lead to spatial heterogeneity in its mechanical properties. In this study we seek to quantify the mechanical heterogeneity within malignant and benign tumors using ultrasound based elasticity imaging. By creating in-vivo elastic modulus images for ten human subjects with breast tumors, we show that Young's modulus distribution in cancerous breast tumors is more heterogeneous when compared with tumors that are not malignant, and that this signature may be used to distinguish malignant breast tumors. Our results complement the view of cancer as a heterogeneous disease on multiple length scales by demonstrating that mechanical properties within cancerous tumors are also spatially heterogeneous.