We report the design, fabrication and characterization of novel dichroic mirrors embedded in a tightly confining AlGaAs/Al(x)O(y) waveguide. Reflection at the first-harmonic wavelength as high as 93% is achieved, while high transmission is maintained at the second-harmonic wavelength. The measured cavity spectrum is in excellent agreement with finite-difference time-domain simulations. Such a mirror is essential for achieving resonant enhancement of second-harmonic generation.