Recent studies on yeast transcriptome have revealed the presence of a large set of RNA polymerase II transcripts mapping to intergenic and antisense regions or overlapping canonical genes. Most of these ncRNAs (ncRNAs) are subject to termination by the Nrd1-dependent pathway and rapid degradation by the nuclear exosome and have been dubbed cryptic unstable transcripts (CUTs). CUTs are often considered as by-products of transcriptional noise, but in an increasing number of cases they play a central role in the control of gene expression. Regulatory mechanisms involving expression of a CUT are diverse and include attenuation, transcriptional interference, and alternative transcription start site choice. This review focuses on the impact of cryptic transcription on gene expression, describes the role of the Nrd1-complex as the main actor in preventing nonfunctional and potentially harmful transcription, and details a few systems where expression of a CUT has an essential regulatory function. We also summarize the most recent studies concerning other types of ncRNAs and their possible role in regulation.