Various four-mirror optical resonators are studied from the perspective of realizing passive stacking cavities. A comparative study of the mechanical stability is provided. The polarization properties of the cavity eigenmodes are described, and it is shown that the effect of mirror misalignments (or motions) induces polarization and stacking power instabilities. These instabilities increase with the finesse of the Fabry-Perot cavity. A tetrahedral configuration of the four mirrors is found to minimize the consequences of the mirrors' motion and misalignment by reducing the instability parameter by at least 2 orders of magnitude.