There is a considerable disagreement about the extent to which solutes perturb water structure. On the one hand, studies that analyse structure directly only show local structuring in a solute's first and possibly second hydration shells. On the other hand, thermodynamic and kinetic data imply indirectly that structuring occurs much further away. Here, the hydrogen-bond structure of water around halide anions, alkali cations, noble-gas solutes, and at the vapor-water interface is examined using molecular dynamics simulations. In addition to the expected perturbation in the first hydration shell, deviations from bulk behavior are observed at longer range in the rest of the simulation box. In particular, at the longer range, there is an excess of acceptors around halide anions, an excess of donors around alkali cations, weakly enhanced tetrahedrality and an oscillating excess and deficiency of donors and acceptors around noble-gas solutes, and enhanced tetrahedrality at the vapor-water interface. The structuring compensates for the short-range perturbation in water-water hydrogen bonds induced by the solute. Rather than being confined close to the solute, it is spread over as many water molecules as possible, presumably to minimize the perturbation to each water molecule.