In this paper, we describe the use of a temperature responsive polymer to reversibly assemble gold nanoparticles of various sizes. Temperature responsive, low critical solution temperature (LCST) pNIPAAm-co-pAAm polymers, with transition temperatures (T(C)) of 51 and 65 °C, were synthesized with a thiol modification, and grafted to the surface of 11 and 51 nm gold nanoparticles (AuNPs). The thermal-responsive behavior of the polymer allowed for the reversible aggregation of the nanoparticles, where at T < T(C) the polymers were hydrophilic and extended between particles. In contrast, at T > T(C), the polymer shell undergoes a hydrophilic to hydrophobic phase transition and collapses, decreasing interparticle distances between particles, allowing aggregation to occur. The AuNP morphology and polymer conjugation were probed by TEM, FTIR, and (1)H NMR. The thermal response was probed by UV-vis and DLS. The structure of the assembled aggregates at T > T(C) was studied via in situ small-angle X-ray scattering, which revealed interparticle distances defined by polymer conformation.