🧭 𝐂𝐡𝐨𝐨𝐬𝐢𝐧𝐠 𝐚𝐧 𝐞𝐦𝐛𝐞𝐝𝐝𝐢𝐧𝐠 𝐢𝐧𝐟𝐞𝐫𝐞𝐧𝐜𝐞 𝐬𝐨𝐥𝐮𝐭𝐢𝐨𝐧
for open models
These days we have many good open embedding models - think for example of the models released by mixedbread.ai a few days ago.
There are also several libraries to use/serve them.
Navigating this landscape can be complex, so let's explore together (thanks to Luca Santuari for the question). 👇
⭐ 𝐒𝐞𝐧𝐭𝐞𝐧𝐜𝐞 𝐓𝐫𝐚𝐧𝐬𝐟𝐨𝐫𝐦𝐞𝐫𝐬
A cornerstone library for computing text embeddings. Most of the embedding models available on Hugging Face are compatible with it.
Originally developed by Ubiquitous Knowledge Processing (UKP) Lab and Nils Reimers, it was recently revamped by HuggingFace and is maintained by Tom Aarsen. 💙
Python library, depends on PyTorch, may not be the most efficient and fast. Runs best on 𝐆𝐏𝐔.
🚀 𝐇𝐅 𝐓𝐞𝐱𝐭 𝐄𝐦𝐛𝐞𝐝𝐝𝐢𝐧𝐠𝐬 𝐈𝐧𝐟𝐞𝐫𝐞𝐧𝐜𝐞
Toolkit for deploying and serving open-source text embedding models.
Very fast and efficient: based on the Rust candle framework. Runs via docker and supports both 𝐂𝐏𝐔 and 𝐆𝐏𝐔.
Compatible with several Sentence Transformers model architectures.
Not fully liberal open source license.
🤗 𝐇𝐅 𝐎𝐩𝐭𝐢𝐦𝐮𝐦
This project is an extension of Transformers that provides a set of performance optimization tools to train and run models with maximum efficiency.
It supports different specialized hardware options from various vendors (Nvidia, Intel...) and the cross-platform ONNX runtime.
This toolkit can also be used to compute embeddings with different and efficient 𝐂𝐏𝐔 and 𝐆𝐏𝐔 options.
⚡️ 𝐅𝐚𝐬𝐭𝐄𝐦𝐛𝐞𝐝
Originally developed by Nirant Kasliwal and maintained by Qdrant, this library provides fast and efficient embedding generation. Easy to use.
It is based on the ONNX runtime and runs on 𝐂𝐏𝐔. Supports a limited but growing selection of models.
🦙 𝐎𝐥𝐥𝐚𝐦𝐚
Very popular library for LLM serving on standard machines, using the GGUF quantized format.
Recently improved support for embedding models. It uses 𝐂𝐏𝐔 + 𝐆𝐏𝐔 if available.
The embedding functionality is still immature compared to previous solutions, but it might make sense if you already use it for Generative Language Models.
You know what? 😉 All of these solutions are supported by the #haystack LLM framework 👉 https://lnkd.in/d8thsQ9a
A special mention goes to ♾️ Infinity, by Michael Feil, which I have not tried yet, but looks great!
💬 Have I missed significant solutions? What is your experience with these libraries?
Let me know in the comments!
#nlp #naturallanguageprocessing #transformers #llm