We study the dynamics of small vortex clusters with a few (2-4) corotating vortices in Bose-Einstein condensates by means of experiments, numerical computations, and theoretical analysis. All of these approaches corroborate the counterintuitive presence of a dynamical instability of symmetric vortex configurations. The instability arises as a pitchfork bifurcation at sufficiently large values of the vortex system angular momentum that induces the emergence and stabilization of asymmetric rotating vortex configurations. The latter are quantified in the theoretical model and observed in the experiments. The dynamics is explored both for the integrable two-vortex particle system, where a reduction of the phase space of the system provides valuable insight, as well as for the nonintegrable three- (or more) vortex case, which additionally admits the possibility of chaotic trajectories.