We demonstrate the proof of concept of a novel Fourier-domain optical coherence tomography contrast mechanism using gold nanorod contrast agents and a spectral fractionation processing technique. The methodology detects the spectral shift of the backscattered light from the nanorods by comparing the ratio between the short and long wavelength halves of the optical coherence tomography signal intensity. Spectral fractionation further divides the halves into sub-bands to improve spectral contrast and suppress speckle noise. Herein, we show that this technique can detect gold nanorods in intralipid tissue phantoms. Furthermore, cellular labeling by gold nanorods was demonstrated using retinal pigment epithelial cells in vitro.