Small angle neutron and x-ray scattering methods are used to investigate the structure of dilute suspensions of two different ferrofluid systems dispersed in soft polyacrylamide hydrogels. It is found that the particles in the fluid are fractal aggregates composed of smaller particles of radius ca. 5 nm. The fractal dimension is strongly dependent on sample, taking the value 1.7 in the first sample and 2.9 in the second sample. In the presence of a magnetic field the aggregates orient, but are restricted in both their translational and rotational freedom. The effect of the gel elasticity is treated as a hindrance to the orientation process.