Astrometric measurements using stellar interferometry rely on the precise measurement of the central white-light fringe to accurately obtain the optical path-length difference of incoming starlight to the two arms of the interferometer. Because of dispersion in the optical system the optical path-length difference is a function of the wavelength of the light and extracting the proper astrometric signatures requires accommodating these effects. One standard approach to stellar interferometry uses a channeled spectrum to determine phases at a number of different wavelengths that are then converted to the path-length delay. Because of throughput considerations these channels are made sufficiently broad so that monochromatic models are inadequate for retrieving the phase/delay information. The presence of dispersion makes the polychromatic modeling problem for phase estimation even more difficult because of its effect on the complex visibility function. We introduce a class of models that rely on just a few spectral and dispersion parameters. A phase-shifting interferometry algorithm is derived that exploits the model structure. Numerical examples are given to illustrate the robustness and precision of the approach.