The softness of microgels depends on many aspects, such as particle characteristic lengths, sample concentration, chemical composition of the sample, and elastic moduli of the particle. Here, the response to crowding of ionic microgels is studied. Charged and uncharged ionic microgels are studied in concentrated suspensions of both neutral and ionic microgels with the same swollen size. The combination of small-angle X-ray and neutron scattering with contrast variation allows us to probe both the particle-to-particle arrangement and the response of individual ionic microgels to crowding. When the ionic microgels are uncharged, initial isotropic deswelling followed by faceting is observed. Therefore, the ionizable groups in the polymeric network do not affect the response of the ionic microgel to crowding, which is similar to what has been reported for neutral microgels. In contrast, the kind of microgels composing the matrix plays a key role once the ionic microgels are charged. If the matrix is composed of neutral microgels, a pronounced faceting and negligible deswelling is observed. When only charged ionic microgels are present in the suspension, isotropic deswelling without faceting is dominant.