We bring the methodology of orienting polar molecules together with the phase sensitivity of high harmonic spectroscopy to experimentally compare the phase difference of attosecond bursts of radiation emitted upon electron recollision from different ends of a polar molecule. This phase difference has an impact on harmonics from aligned polar molecules, suppressing emission from the molecules parallel to the driving laser field while favoring the perpendicular ones. For oriented molecules, we measure the amplitude ratio of even to odd harmonics produced when intense light irradiates CO molecules and determine the degree of orientation and the phase difference of attosecond bursts using molecular frame ionization and recombination amplitudes. The sensitivity of the high harmonic spectrum to subtle phase differences in the emitted radiation makes it a detailed probe of polar molecules and will drive major advances in the theory of high harmonic generation.