Neural responses are most often characterized in terms of the sets of environmental or internal conditions or stimuli with which their firing rate [corrected]increases or decreases are correlated [corrected] Their transient (nonstationary) temporal profiles of activity have received comparatively less attention. Similarly, the computational framework of attractor neural networks puts most emphasis on the representational or computational properties of the stable states of a neural system. Here we review a couple of neurophysiological observations and computational ideas that shift the focus to the transient dynamics of neural systems. We argue that there are many situations in which the transient neural behaviour, while hopping between different attractor states or moving along 'attractor ruins', carries most of the computational and/or behavioural significance, rather than the attractor states eventually reached. Such transients may be related to the computation of temporally precise predictions or the probabilistic transitions among choice options, accounting for Weber's law in decision-making tasks. Finally, we conclude with a more general perspective on the role of transient dynamics in the brain, promoting the view that brain activity is characterized by a high-dimensional chaotic ground state from which transient spatiotemporal patterns (metastable states) briefly emerge. Neural computation has to exploit the itinerant dynamics between these states.