Airflow tutorial | Install Airflow | Write and run your first DAG | Apache airflow on Windows Docker

Summary

The tutorial demonstrates how to install Apache Airflow on a Windows machine using Docker, then create and run a first DAG. Because Airflow is not officially supported on Windows, the instructor runs it inside Docker containers.

Setup and Configuration

• Install Docker Desktop for Windows and update WSL if needed.
• Install Visual Studio Code, along with its Docker and Python extensions.
• Create a Dockerfile based on the official Apache Airflow image. The example adds Git and leaves room for later customizations.
• Create a docker-compose.yaml file to configure the Airflow services and environment:
  • Mount the local Airflow folder into the container so DAG files can be developed locally.
  • Mount persistent storage so DAG run history and logs are retained when containers restart.
  • Map host port 8080 to container port 8080 to access the Airflow web interface.
• Start the services with Docker Compose. The configuration launches Airflow components such as the scheduler, worker, and web server.
• Log in to the web UI with the default username, admin. The initial password can be found in the container logs or local Airflow folder.

The tutorial uses SQLite for the metadata database, which is suitable for development but not production. PostgreSQL is mentioned as a planned option for a later lesson.

First DAG Tutorial

• Create a dags directory and add welcomedag.py.
• The sample DAG has three tasks:
  1. Print a welcome message.
  2. Display the current date.
  3. Print a random quote from a website.
• Link the tasks with dependencies and schedule the DAG to run daily at 11 PM.
• Allow several minutes for the DAG to appear in the Airflow UI. The instructor manually triggers it for the demonstration, verifies that all three tasks succeed, and reviews their logs.
• Stop the setup with Docker Compose’s compose down command.

Main Speaker/Source

Narrator/instructor: SleekData, presenting the Airflow setup and first-DAG walkthrough.