To gain insight into how ERG translocations cause prostate cancer, we performed single cell transcriptional profiling of an autochthonous mouse model at an early stage of disease initiation. Despite broad expression of ERG in all prostate epithelial cells, proliferation was enriched in a small, stem-like population with mixed-luminal basal identity (called intermediate cells). Through a series of lineage tracing and primary prostate tissue transplantation experiments, we find that tumor initiating activity resides in a subpopulation of basal cells that co-express the luminal genes Tmprss2 and Nkx3.1 (called BasalLum) but not in the larger population of classical Krt8+ luminal cells. Upon ERG activation, BasalLum cells give rise to the highly proliferative intermediate state, which subsequently transitions to the larger population of Krt8+ luminal cells characteristic of ERG-positive human cancers. Furthermore, this proliferative population is characterized by an ERG-specific chromatin state enriched for NFkB, AP-1, STAT and NFAT binding, with implications for TF cooperativity. The fact that the proliferative potential of ERG is enriched in a small stem-like population implicates the chromatin context of these cells as a critical variable for unmasking its oncogenic activity.