The endoplasmic reticulum (ER) must target potentially toxic misfolded proteins for retrotranslocation and proteasomal degradation while avoiding destruction of productive folding intermediates. For luminal proteins, this discrimination typically depends not only on the folding status of a polypeptide, but also on its glycosylation state. Two putative sugar binding proteins, Htm1p and Yos9p, are required for degradation of misfolded glycoproteins, but the nature of the glycan degradation signal and how such signals are generated and decoded remains unclear. Here we characterize Yos9p's oligosaccharide-binding specificity and find that it recognizes glycans containing terminal alpha1,6-linked mannose residues. We also provide evidence in vivo that a terminal alpha1,6-linked mannose-containing oligosaccharide is required for degradation and that Htm1p acts upstream of Yos9p to mediate the generation of such sugars. This strategy of marking potential substrates by Htm1p and decoding the signal by Yos9p is well suited to provide a proofreading mechanism that enhances substrate specificity.