The near K-edge structure of oxygen in liquid water and ices III, II, and IX at 0.25 GPa and several low temperatures down to 4 K has been studied using inelastic x-ray scattering at 9884.7 eV with a total energy resolution of 305 and 175 meV. A marked decrease of the preedge intensity from the liquid phase and ice III to ices II and IX is attributed to ordering of the hydrogen bonds in the proton-ordered lattice of the latter phases. Density functional theory calculations including the influence of the Madelung potential of the ice IX crystal correctly account for the remaining preedge feature. Furthermore, we obtain spectroscopic evidence suggesting a possible new phase of ice at temperatures between 4 and 50 K.