Supernumerary B chromosomes are optional additions to the basic set of A chromosomes, and occur in all eukaryotic groups. They differ from the basic complement in morphology, pairing behavior, and inheritance and are not required for normal growth and development. The current view is that B chromosomes are parasitic elements comparable to selfish DNA, like transposons. In contrast to transposons, they are autonomously inherited independent of the host genome and have their own mechanisms of mitotic or meiotic drive. Although B chromosomes were first described a century ago, little is known about their origin and molecular makeup. The widely accepted view is that they are derived from fragments of A chromosomes and/or generated in response to interspecific hybridization. Through next-generation sequencing of sorted A and B chromosomes, we show that B chromosomes of rye are rich in gene-derived sequences, allowing us to trace their origin to fragments of A chromosomes, with the largest parts corresponding to rye chromosomes 3R and 7R. Compared with A chromosomes, B chromosomes were also found to accumulate large amounts of specific repeats and insertions of organellar DNA. The origin of rye B chromosomes occurred an estimated ∼1.1-1.3 Mya, overlapping in time with the onset of the genus Secale (1.7 Mya). We propose a comprehensive model of B chromosome evolution, including its origin by recombination of several A chromosomes followed by capturing of additional A-derived and organellar sequences and amplification of B-specific repeats.