We investigate a general scenario for "glassy" or "jammed" systems driven by an external, nonconservative force, analogous to a shear force in a fluid. In this scenario, the drive results in the suppression of the usual aging process, and the correlation and response functions become time translation invariant. The relaxation time and the response functions are then dependent on the intensity of the drive and on temperature. We investigate this dependence within the framework of a dynamical closure approximation that becomes exact for disordered, fully connected models. The relaxation time is shown to be a decreasing function of the drive ("shear thinning" effect). The correlation functions below the glass transition temperature (Tc) display a two-time-scale relaxation pattern, similar to that observed at equilibrium slightly above Tc. We also study the violation of the fluctuation-dissipation relationship in the driven system. This violation is very reminiscent of the one that takes place in a system aging below Tc at zero drive. It involves, in particular the appearance of a two-temperature regime, in the sense of an effective fluctuation-dissipation temperature [L. F. Cugliandolo, J. Kurchan, and L. Peliti, Phys. Rev. E 55, 3898 (1997)]. Although our results are, in principle, limited to the closure relations that hold for mean-field models, we argue that a number of the salient features are not inherent to the approximation scheme, and may be tested in experiments and simulations.