Brev.dev

How do LLM inference optimizations work? I had this question so I sat down with the legend, Kyle Kranen to learn more about them. NVIDIA is full of experts on anything AI related, is there something you want me to ask? I can even mic up Kyle again 😂❤️🤙

What is the most SF tech scene thing you can think of? And why is it physical A100-hour gift cards? Thank you to Nader Khalil and Brev.dev (acquired by NVIDIA) for providing me with the best party favor of all time. What would you do with a couple of these?

What is the most SF tech scene thing you can think of? And why is it physical A100-hour gift cards? Thank you to Nader Khalil and Brev.dev (acquired by NVIDIA) for providing me with the best party favor of all time. What would you do with a couple of these?

Usually Twitter is the place to break AI news, but today LinkedIn goes first because enterprises need to hear this news about us self-hosting Llama3 405B in an evening! I am thrilled to announce that Travis Cline, Sagar Saija, Saurav Panda & I successfully got Llama3 405B up and running in just 2 hours on our own 8xA100 machine last night at AGI House Global SF's Emergency Llama3 Hackathon! We load-tested our self-hosted endpoint and got up to 1000 tokens / second. This is crazy! I'm running a model as powerful as ChatGPT / GPT-4o ... not quite at home but ... on a rented machine for $20K per month. THIS IS INSANE! AND AWESOME! It's like OpenAI, Google, Anthropic and .... *checks notes* ... me. Next up is my dream project ... fully OSS self-hosted scalable CODE INTERPRETER. In a democratic twist, the judges for last night were the Open Source hackers themselves. We snagged 🥇 first place. Sponsors included Together AI, Lepton AI & our particular project was sponsored by Brev.dev, who have really great tooling for exactly this kind of thing. I also had a great experience running the model on OctoAI while testing! Shoutout to vLLM really.

Hack AI is tomorrow night, August 1st, at 6:00 PM 🤠 We will be fine-tuning and deploying Llama 3.1, Meta's latest open-source Large Language Model. Compute is covered thanks to Brev.dev! And we will have Food, Drinks, and Prizes. If you're a developer, or just interested in learning more about AI, come by the 8th floor of Capital Factory tomorrow night! Learn More: https://lnkd.in/g6A2929H

There are a lot of improvements coming soon for this project, but if you want to run Apache Lucene on GPUs to do some eye-popping things (benchmarks in the repo), a few teammates and I wrote a tutorial for how to compile and use Lucene on Brev.dev GPUs. They have a ton of Nvidia GPUs. https://lnkd.in/gK5--f2n Did somebody say query, index, inference on the same compute? Stay tuned for a very fast rewrite of this integration to use different native APIs, followed by a vector DB or two on GPUs. 🚀🚀 I hope that people will build more cool projects using specialized hardware when we can show them how cost effective they could be. For those don't know what Apache Lucene is, you're using it right now because you're on LinkedIn, where it's one of the largest deployments for search and content discovery. :-) Call me if you have any trouble with the tutorial in the repo: 510-495-5257

Today Brev.dev is being acquired by NVIDIA!  Brev’s goal is to build the easiest way for AI/ML developers to use a GPU. Teaming up with NVIDIA means being able to deliver on that mission by pairing the most powerful hardware with the industry leading software.  Thanks to all of our users who have been with us thus far in the Brev Journey, we’re excited to bring you along for the next chapter. Our next release is coming soon! Stay tuned and Let it Rip 🤙

My Brev.dev Origin Story. Sadly the Arabic didn’t stick! I blame Nader Khalil

Metas new model Chameleon blew my mind 🤯 It's multi-modal, so text and images, but it does so using the same encoder! Unlike image gen models, it uses tokens not diffusion! To do this, it has to convert the text and images into a single sequence of tokens when processing. Then, a single transformer processes these mixed-modal tokens, eliminating the need for a different encoder per modality. If this works well, it might make for more scalable and efficient multi-modal models! I'll put a link to run it in the comments 🤙 Note: they made it "safety-aligned" and ripped out the ability to generate images, however, that code is still in there. If you are able to unlock it, I'll give you $1000

Watch the demo, then run it yourself with Brev.dev