Cloud Composer: Qwik Start - Command Line

In this lab, you use Cloud Shell command line to create the Cloud Composer environment and set Composer environment variables. You then use Composer to run a workflow that verifies a data file, creates and runs an Apache Hadoop wordcount job on a Dataproc cluster, and deletes the cluster.

• GSP606
• Overview
• Setup and requirements
• What is Cloud Composer?
• What is Apache Airflow?
• What is Cloud Dataproc?
• Task 1. Sample workflow
• Task 2. Verify the Composer environment
• Task 3. Set up Apache Airflow environment variables
• Task 4. Upload Airflow files to Cloud storage
• Task 5. Using the Airflow web interface
• Delete Cloud Composer Environment
• Congratulations!