For both Fourier and non-Fourier moving patterns, models have been proposed which detect motion based on either the net orientation of energy in the stimulus (after nonlinear stage for non-Fourier motion stimuli) or on the changes in the relative locations of spatial primitives in the image. Both approaches have been successful in accounting for detection of simple translational displacements, but we examined how such models coped with more demanding stimuli. We examined direction discrimination using two-flash random Gabor kinematograms which selectively reveal Fourier and non-Fourier motion mechanisms. In addition to target elements, multiple distractor elements were added, either static or randomly moving. It was found that detection of Fourier motion was relatively unaffected by the distractors unless they were of orthogonal orientation. Detection of non-Fourier motion was possible, but with a slightly higher error rate, even with many distractors and was not at all affected by orthogonal distractors. The results for distractors of the same orientation as targets are in better agreement with predictions of energy than with edge-matching models. The differing effects of orthogonal distractors further strengthen the proposed dichotomy of quasi-linear and nonlinear motion mechanisms, but indicate that the latter operates on a more complex representation than a simple contrast envelope.