The accumulation of compatible solutes, either by uptake from the medium or by de novo synthesis, is a general response of microorganisms to osmotic stress. The diversity of compatible solutes is large but falls into a few major chemical categories, such as carbohydrates or their derivatives and amino acids or their derivatives. This review deals with compatible solutes found in thermophilic or hyperthermophilic bacteria and archaea that have not been commonly identified in microorganisms growing at low and moderate temperatures. The response to NaCl stress of Thermus thermophilus is an example of how a thermophilic bacterium responds to osmotic stress by compatible solute accumulation. Emphasis is made on the pathways leading to the synthesis of mannosylglycerate and glucosylglycerate that have been recently elucidated in several hyper/thermophilic microorganisms. The role of compatible solutes in the thermoprotection of these fascinating microorganisms is also discussed.