Time-resolved images of the magnetization switching process in a spin transfer structure, obtained by ultrafast x-ray microscopy, reveal the limitations of the macrospin model. Instead of a coherent magnetization reversal, we observe switching by lateral motion of a magnetic vortex across a nanoscale element. Our measurements reveal the fundamental roles played independently by the torques due to charge and spin currents in breaking the magnetic symmetry on picosecond time scales.