The nucleolus is a nuclear organelle that coordinates rRNA transcription and ribosome subunit biogenesis. Recent proteomic analyses have shown that the nucleolus contains proteins involved in cell cycle control, DNA processing and DNA damage response and repair, in addition to the many proteins connected with ribosome subunit production. Here we study the dynamics of nucleolar protein responses in cells exposed to stress and DNA damage caused by ionizing and ultraviolet (UV) radiation in diploid human fibroblasts. We show using a combination of imaging and quantitative proteomics methods that nucleolar substructure and the nucleolar proteome undergo selective reorganization in response to UV damage. The proteomic responses to UV include alterations of functional protein complexes such as the SSU processome and exosome, and paraspeckle proteins, involving both decreases and increases in steady state protein ratios, respectively. Several nonhomologous end-joining proteins (NHEJ), such as Ku70/80, display similar fast responses to UV. In contrast, nucleolar proteomic responses to IR are both temporally and spatially distinct from those caused by UV, and more limited in terms of magnitude. With the exception of the NHEJ and paraspeckle proteins, where IR induces rapid and transient changes within 15 min of the damage, IR does not alter the ratios of most other functional nucleolar protein complexes. The rapid transient decrease of NHEJ proteins in the nucleolus indicates that it may reflect a response to DNA damage. Our results underline that the nucleolus is a specific stress response organelle that responds to different damage and stress agents in a unique, damage-specific manner.