From the course: Hands-On Introduction: Data Engineering

Schalten Sie den kompletten Kurs noch heute frei

Join today to access over 23,400 courses taught by industry experts.

Running the Airflow webserver and scheduler

Running the Airflow webserver and scheduler

From the course: Hands-On Introduction: Data Engineering

Start my 1-month free trial Buy for my team

Running the Airflow webserver and scheduler

"

- [Instructor] Airflow is both a library and an application, and in this context, we'll be setting up the application side of Airflow. We'll cover the two key components of Airflow, the web server, and the scheduler, and their respective roles in the Airflow ecosystem. We'll also discuss how to start and stop them. The Airflow Webserver is a web-based UI that is commonly used in production to provide an overview of all DAGs and their execution flow. It offers several ways to manage administrative settings, connections, variables, and other components of Airflow through a web interface. The web server is also capable of responding to HTTP requests and it can be configured with different viewer and editor modes. It is worth noting that the web server is not a required component of the Airflow ecosystem and can be managed entirely from the command line, although this may not be the most user-friendly option. The Airflow Scheduler is a process that continually monitors all tasks and DAGs…

Inhalt