From the course: Responsible Generative AI and Local LLMs
Unlock this course with a free trial
Join today to access over 23,300 courses taught by industry experts.
Exploring remote development on AWS
From the course: Responsible Generative AI and Local LLMs
Exploring remote development on AWS
- [Instructor] Let's take a look at a Rust and Hugging Face Candle asynchronous development workflow. How would that look? First up, you have an EC2 G5 instance here. What's great about that is it has the power to do modern GPU inference and you can actually use spot instances as well to save money. But in development, what you can do is actually use an instance you stop and start, and you can connect to that instance via Visual Studio Code Remote, which is a great way to develop. Really just install the plugin, you're ready to go, and your laptop can have access to that. So what it means is that you can start developing code into a cloud-based environment that's very powerful and you can use it and pay for what you use versus buying, let's say a $40,000 GPU. Now, in terms of production, here's where things get interesting. You could also be co-developing with EFS. So the mount point could sit onto your development box, but that same mount point could also be used for production. You…
Inhalt
-
-
Coding ELO in Python4m 7s
-
Coding ELO in Rust3m 49s
-
Coding ELO in R3m 31s
-
(Locked)
Coding ELO in Julia3m 5s
-
(Locked)
Profit sharing concepts5m 40s
-
(Locked)
Tragedy of the commons4m
-
(Locked)
Deploying LLMs with Lorax and SkyPilot3m 56s
-
(Locked)
Fine-tune Mistral and Ludwig3m 22s
-
(Locked)
Game theory in generative AI4m 45s
-
(Locked)
Perfect competition2m 45s
-
(Locked)
Negative externalities3m 23s
-
(Locked)
Regulatory entrepreneurship4m 18s
-
(Locked)
Creating reinforcement bias3m 59s
-
(Locked)
Getting started with Mozilla llamafile3m 36s
-
(Locked)
Developing cosmopolitan4m 29s
-
(Locked)
Building blocks for generative AI with whisper.cpp2m 53s
-
(Locked)
Transcribing with Whisper2m 56s
-
(Locked)
Portable phrase CLI3m 34s
-
(Locked)
Candle hello world2m 56s
-
(Locked)
Exploring StarCoder in Rust5m 54s
-
(Locked)
Whisper Candle transcriber5m 51s
-
(Locked)
Local system metrics3m 4s
-
(Locked)
Exploring remote development on AWS2m 15s
-
(Locked)
Rust for large language models (LLMs)2m
-
(Locked)
The continuous build binary2m 6s
-
(Locked)
Serverless inference1m 56s
-
(Locked)
Rust CLI inference2m 7s
-
(Locked)
Rust chat inference2m 3s
-
(Locked)
The chat loop with StarCoder2m 4s
-
(Locked)
Invoke an LLM on an AWS G5 instance, part 14m 36s
-
(Locked)
Invoke an LLM on an AWS G5 instance, part 22m 58s
-