Extracellular vesicles (EVs) have emerged as promising diagnostic and therapeutic candidates in many biomedical applications. However, EV research continues to rely heavily on in vitro cell cultures for EV production, where the exogenous EVs present in fetal bovine (FBS) or other required serum supplementation can be difficult to remove entirely. Despite this and other potential applications involving EV mixtures, there are currently no rapid, robust, inexpensive, and label-free methods for determining the relative concentrations of different EV subpopulations within a sample. In this study, we demonstrate that surface-enhanced Raman spectroscopy (SERS) can biochemically fingerprint fetal bovine serum-derived and bioreactor-produced EVs, and after applying a novel manifold learning technique to the acquired spectra, enables the quantitative detection of the relative amounts of different EV populations within an unknown sample. We first developed this method using known ratios of Rhodamine B to Rhodamine 6G, then using known ratios of FBS EVs to breast cancer EVs from a bioreactor culture. In addition to quantifying EV mixtures, the proposed deep learning architecture provides some knowledge discovery capabilities which we demonstrate by applying it to dynamic Raman spectra of a chemical milling process. This label-free characterization and analytical approach should translate well to other EV SERS applications, such as monitoring the integrity of semipermeable membranes within EV bioreactors, ensuring the quality or potency of diagnostic or therapeutic EVs, determining relative amounts of EVs produced in complex co-culture systems, as well as many Raman spectroscopy applications.