Diffusion of tracer particles in the cytoplasm of mammalian cells is often anomalous with a marked heterogeneity even within individual particle trajectories. Despite considerable efforts, the mechanisms behind these observations have remained largely elusive. To tackle this problem, we performed extensive single-particle tracking experiments on quantum dots in the cytoplasm of living mammalian cells at varying conditions. Analyses of the trajectories reveal a strong, microtubule-dependent subdiffusion with antipersistent increments and a substantial heterogeneity. Furthermore, particles stochastically switch between different mobility states, most likely due to transient associations with the cytoskeleton-shaken endoplasmic reticulum network. Comparison to simulations highlight that all experimental observations can be fully described by an intermittent fractional Brownian motion, alternating between two states of different mobility.