We report a straightforward way for forming and tuning the optical properties of thermally responsive plasmonic nanogels. Upon functionalization, a small red shift (2-3 nm) of the pNIPAM@AuNPs was observed due to changes in the refractive index surrounding the AuNP. By adding thermoresponsive poly-N-isopropylacrylamide (pNIPAM) into the pNIPAM@AuNP, its optical response was significantly increased. Heating the nanogel such that the pNIPAM collapsed and acted as a cross-link resulted in the aggregation of the AuNPs. The plasmonic response with red shifts of up to 20 nm was observed. The enlarged red shift was due to the increase in the dielectric constant around the particles and the interparticle interaction of the AuNPs. The interparticle interaction also leads to the broadening of the spectra. Experimental data and finite-difference time-domain (FDTD) calculation are in agreement with this observation. The temperature-dependent optical properties were reversible through multiple cycles of heating and cooling.