Ultrashort surface plasmon polariton (SPP) pulses, propagating on the surface of a nanostructured metallic film, are characterized in space and time using time-resolved spatial-heterodyne imaging. Optical pulses are coupled from free space into various surface modes using a 2D array of circular nanoholes, and spatial amplitude and phase characteristics of the scattered surface field are measured with femtosecond-scale time resolution. Demonstrated in-plane focusing of SPP pulse provides additional electromagnetic field localization with possible applications in SPP nanophotonics, nonlinear surface dynamics, biochemical sensing, and ultrafast surface studies.