We have observed a novel magnetic structure in the pseudomorphic Fe monolayer on Ir(111). Using spin-polarized scanning tunneling microscopy we find a nanometer-sized two-dimensional magnetic unit cell. A collinear magnetic structure is proposed consisting of 15 Fe atoms per unit cell with 7 magnetic moments pointing in one and 8 moments in the opposite direction. First-principles calculations verify that such an unusual magnetic state is indeed lower in energy than all solutions of the classical Heisenberg model. We demonstrate that the complex magnetic structure is induced by the strong Fe-Ir hybridization.