Cholera toxin (CT) travels from the plasma membrane of intestinal cells to the endoplasmic reticulum (ER) where a portion of the A-subunit, the A1 chain, crosses the membrane into the cytosol to cause disease. A related toxin, LTIIb, binds to intestinal cells but does not cause toxicity. Here, we show that the B-subunit of CT serves as a carrier for the A-subunit to the ER where disassembly occurs. The B-subunit binds to gangliosides in lipid rafts and travels with the ganglioside to the ER. In many cells, LTIIb follows a similar pathway, but in human intestinal cells it binds to a ganglioside that fails to associate with lipid rafts and it is sorted away from the retrograde pathway to the ER. Our results explain why LTIIb does not cause disease in humans and suggest that gangliosides with high affinity for lipid rafts may provide a general vehicle for the transport of toxins to the ER.