From the course: Responsible Generative AI and Local LLMs
Unlock this course with a free trial
Join today to access over 23,300 courses taught by industry experts.
Getting started with Mozilla llamafile
From the course: Responsible Generative AI and Local LLMs
Getting started with Mozilla llamafile
- [Instructor] Here we have the llamafile project from Mozilla, probably the easiest possible way to run a local large language model. Many people are interested in running large language models locally because of the privacy aspects, and also it's free, right? If you are able to use your own machine, you can dive right into the llamafile here. So let's go ahead and take a look at the structure of this project first. So you can see that llamafile lets you distribute and run LLMs with a single file. And what's really fascinating about this is because they use this library, Cosmopolitan Libc, it is able to collapse everything into a single file, executable here called "llamafile." And it's actually, even though it's called "llamafile," it actually is not really tied to any particular large language model. There's a example here where you could download LLaVA, which is able to do reading of images, which is a pretty cool project if you want to play around with that. The one that I think…
Inhalt
-
-
Coding ELO in Python4m 7s
-
Coding ELO in Rust3m 49s
-
Coding ELO in R3m 31s
-
(Locked)
Coding ELO in Julia3m 5s
-
(Locked)
Profit sharing concepts5m 40s
-
(Locked)
Tragedy of the commons4m
-
(Locked)
Deploying LLMs with Lorax and SkyPilot3m 56s
-
(Locked)
Fine-tune Mistral and Ludwig3m 22s
-
(Locked)
Game theory in generative AI4m 45s
-
(Locked)
Perfect competition2m 45s
-
(Locked)
Negative externalities3m 23s
-
(Locked)
Regulatory entrepreneurship4m 18s
-
(Locked)
Creating reinforcement bias3m 59s
-
(Locked)
Getting started with Mozilla llamafile3m 36s
-
(Locked)
Developing cosmopolitan4m 29s
-
(Locked)
Building blocks for generative AI with whisper.cpp2m 53s
-
(Locked)
Transcribing with Whisper2m 56s
-
(Locked)
Portable phrase CLI3m 34s
-
(Locked)
Candle hello world2m 56s
-
(Locked)
Exploring StarCoder in Rust5m 54s
-
(Locked)
Whisper Candle transcriber5m 51s
-
(Locked)
Local system metrics3m 4s
-
(Locked)
Exploring remote development on AWS2m 15s
-
(Locked)
Rust for large language models (LLMs)2m
-
(Locked)
The continuous build binary2m 6s
-
(Locked)
Serverless inference1m 56s
-
(Locked)
Rust CLI inference2m 7s
-
(Locked)
Rust chat inference2m 3s
-
(Locked)
The chat loop with StarCoder2m 4s
-
(Locked)
Invoke an LLM on an AWS G5 instance, part 14m 36s
-
(Locked)
Invoke an LLM on an AWS G5 instance, part 22m 58s
-