We present experimental data on the direct enstrophy cascade in decaying two-dimensional turbulence. Velocity and vorticity fields are obtained using particle tracking velocimetry. From those fields we directly compute the enstrophy and energy flux by using a filtering technique inspired by large-eddy simulations. This allows considerable insight into the physical processes of turbulence when compared with structure-function or spectral analysis. The direct cascade of enstrophy is weakly forward, with almost as much backscatter as down-scale enstrophy transfer, whereas the inverse energy cascade is strongly upscale with a modest amount of backscatter.