Genetic variation on the Y chromosome has not been convincingly implicated in prostate cancer risk. To comprehensively analyze the role of inherited Y chromosome variation in prostate cancer risk in individuals of European ancestry, we genotyped 34 binary Y chromosome markers in 3,995 prostate cancer cases and 3,815 control subjects drawn from four studies. In this set, we identified nominally significant association between a rare haplogroup, E1b1b1c, and prostate cancer in stage I (P = 0.012, OR = 0.51; 95% confidence interval 0.30-0.87). Population substructure of E1b1b1c carriers suggested Ashkenazi Jewish ancestry, prompting a replication phase in individuals of both European and Ashkenazi Jewish ancestry. The association was not significant for prostate cancer overall in studies of either Ashkenazi Jewish (1,686 cases and 1,597 control subjects) or European (686 cases and 734 control subjects) ancestry (P(meta) = 0.078), but a meta-analysis of stage I and II studies revealed a nominally significant association with prostate cancer risk (P(meta) = 0.010, OR = 0.77; 95% confidence interval 0.62-0.94). Comparing haplogroup frequencies between studies, we noted strong similarities between those conducted in the US and France, in which the majority of men carried R1 haplogroups, resembling Northwestern European populations. On the other hand, Finns had a remarkably different haplogroup distribution with a preponderance of N1c and I1 haplogroups. In summary, our results suggest that inherited Y chromosome variation plays a limited role in prostate cancer etiology in European populations but warrant follow-up in additional large and well characterized studies of multiple ethnic backgrounds.