A novel imaging technique that produces accurate amplitude and phase images of an optical fiber facet using only a phase-only liquid-crystal on silicon (LCOS) spatial light modulator (SLM) and a single-pixel detector is presented. The system can take images in two orthogonal polarizations and so provides a powerful tool for modal characterization of multimode fibers, which is of increasing importance due to their burgeoning use in telecommunications and medical applications. This technique first uses a simulated annealing algorithm to compute a hologram that collects light from a small region of the fiber facet. Next, the fiber facet is automatically brought into focus using adaptive aberration correction on the SLM. Finally, a common-path interferometer is created using the SLM, and the phase of the optical field at each pixel is determined. Finally, high-definition amplitude and phase images of a ring-core refractive index fiber are presented as a proof-of-principle demonstration of the technique.