An Introductory Course on GCP and Snowflake
The program is focussed on providing a high level introduction to Google Cloud Platform (GCP) and Snowflake. The participants will get an overview of various services of GCP and Snowflake related to Data Engineering and Data Science. Which one to choose when? Also the participants will understand various approaches of Migration and how to build end to end pipelines on GCP.
The intended audience for this course
- Data Engineers
GCP Overview : Holistic View, Architectures, Products and Pipelines
- Why Google Cloud Platform?
- Innovations at Google
- Various Types of Computing on GCP
- Data Lake on Cloud vs on-prem
- How Key Google Products fit in Enterprise Architecture?
- Quick recap of Lambda Architecture
- How to design modern Data Analytics Pipeline on GCP?
- GCP Reference Architectures
- Ways to interact with GCP
- Demonstration of Google Cloud Console
Artificial Intelligence using Google
- Overview of AI
- Ways of Performing Artificial Intelligence
- AI Capabilities of GCP
- Demonstration of some of the AI Platform features
Key Storage Frameworks
- Factors to consider while comparing Storage frameworks
- Google Cloud Storage Overview
- HDFS vs Google Cloud Storage
- Google Cloud Storage vs BigTable
- BigTable vs BigQuery
- BigTable vs FireStore
- Which one to choose when?
- Hands-on Demo on Google Cloud Storage
Key Data Ingestion & Processing Frameworks
- Approach and Mindset shift from on-prem to cloud
- What and Why of
- Optimizing cost
- Pricing discussion
- Demo of DataProc
- Demo of Dataflow
- Demo of Pub/Sub
Key Data Analysis Frameworks
- Hive vs BigQuery
- Building DataLakes using BigQuery
- Salient Features of BigQuery
- BigQuery Pricing
- Demo of BigQuery
Migration to GCP
- Strategies of Migrating to GCP
- Storage Migration Options
- Lift and Shift Approach
- Lift and Re-architect Approach
- Life and Transform Approach
- Choosing products while moving to GCP
- Workflow Orchestration
- What is Cloud Composer?
- Architecture of Cloud Composer
- What and Why of Airflow?
- Airflow vs Oozie
- Interesting Capabilities of Airflow
- Pricing of Cloud Composer
- Demo of Airflow
Building Pipelines on GCP
- Various ways to build pipelines
- What and Why of Data Fusion?
- DataFusion Pricing Discussion
- Demo of DataFusion
- Evolution of Data Warehousing Technologies
- What’s missing in Hive?
- What is Snowflake?
- Why Snowflake?
- Snowflake vs Big Query vs RedShift
- Characteristics of Modern Data Pipeline
- Key Concepts
- Snowflake Architecture
- Partner Ecosystem of Snowflake
- Ingesting Data into Snowflake
- Ways to interact with Snowflake
- Understanding of basics of Cloud Computing terms and concepts