Join us for a FREE hands-on Meetup webinar on From Idea to Impact: Product Management in the Age of Generative AI | Friday, December 13th, 2024 · 5:00 PM IST/ 7:30 AM EST Join us for a FREE hands-on Meetup webinar on From Idea to Impact: Product Management in the Age of Generative AI | Friday, December 13th, 2024 · 5:00 PM IST/ 7:30 AM EST
Search
Close this search box.
Search
Close this search box.

Data Engineering on Microsoft Azure Cloud

Comprehensive Training for Building and Managing Data Pipelines

Duration

3 Days (8 hours per day)

Level

Intermediate Level

Design and Tailor this course

As per your team needs

Edit Content

Join us for an immersive hands-on course on Data Engineering on Microsoft Azure Cloud, where you’ll master the skills necessary to design, implement, and manage data pipelines using Azure’s powerful tools. This comprehensive training covers essential concepts such as ETL/ELT, Data Warehousing, and Data Lakes. The course will also dive deep into Azure’s suite of services.

You’ll also learn how to build and optimize data pipelines, analyze data in relational warehouses, and leverage Azure’s advanced analytics capabilities, including real-time data stream analysis. The course also covers data governance, ensuring you understand how to classify, label, and manage data securely. Whether you’re new to Azure or looking to enhance your data engineering skills, this hands-on course provides the hands-on experience and insights you need to succeed in the rapidly evolving field of data engineering.

 

Edit Content
  • Data Engineers
  • Data Scientists
  • Data Analysts
  • Cloud Engineers
  • IT Professionals transitioning to data engineering roles
  • Anyone interested in data engineering on Azure 
Edit Content
  • What is Data Engineering?
  • Importance and role of data engineers in the data lifecycle
  • Key concepts: ETL, ELT, Data Warehousing, Data Lakes
  • Overview of Azure Cloud
  • Key services and components relevant to data engineering
  • Azure Resource Management: Subscriptions, Resource Groups, and Policies
  • Implementing Data Engineering Pipelines on Azure
  • Introduction to Azure Storage: Blobs, Tables, Queues, and Files
  • Setting up Azure Blob Storage
  • Blob Storage tiers (Hot, Cool, Archive)
  • Blob Types – Block Blobs, Append Blobs, Page Blobs
  • Understanding Azure Data Lake Storage Gen2
  • Enable Azure Data Lake Storage Gen2
  • Azure Data Lake Storage Gen2 vs Azure Blob Storage
  • Understand the stage for processing Big Data
  • Using Azure Data Lake Storage Gen2 in Data Analytics Workloads
  • Overview of Azure Data Factory
  • Key features and components
  • Pipelines, Activities, and Datasets
  • Triggers and scheduling
  • Data Integration and Orchestration
  • Building your first pipeline
  • Connecting to data sources
  • Data movement and transformation
  • Various Data Warehouse options
  • Design a Data Warehouse Schema
  • Create Data Warehouse Tables
  • Load and Query Data Warehouses
  • Introduction to Azure Synapse Analytics
  • Key features and architecture
  • Synapse SQL Pools and Apache Spark Pools
  • Setting up a Synapse workspace
  • Data ingestion into Synapse
  • Querying and analyzing data
  • Understanding Azure Synapse Analytics SQL pool capabilities and use cases
  • Query files using a serverless SQL Pool
  • Include data transformations stored procedures in a pipeline
  • Creating lake databases
  • Managing users within Azure Synapse serverless pool
  • Using Spark in Azure Synapse Analytics
  • Modify and save dataframes
  • Transform data with Spark and SQL in Azure Synapse Analytics
  • Creating and working with Delta Lakes
  • Working with Synapse Notebooks and Pipelines
  • Get started with Azure Databricks
  • Understanding key concepts
  • Ingest and Analyze data within Azure Databricks
  • Understanding Azure Databricks notebooks and pipelines
  • Create a linked services for Azure Databricks
  • Integrating Azure Databricks with Azure Data Factory
  • What is Data Stream and Event Processing?
  • Stream Ingestion Scenarios
  • Configure Inputs and Outputs
  • Ingest streaming data into Stream Analytics
  • Use PowerBI output in Stream Analytics
  • Create a query for Real-Time Visualization
  • Real-Time Visualization within PowerBI
  • What is Azure Purview?
  • How does Azure Purview work?
  • Search and Browse Assets
  • Register and Scan Data
  • Classify and Label Data
  • Integrating with Azure Synapse Analytics
  • View PowerBI metadata and lineage
Edit Content

Before attending this Azure Data Engineering workshop, participants should have the following prerequisites to ensure they can keep up with the pace and content of the course:

  • Basic Understanding of Data Concepts
  • Experience with Databases
  • Fundamental Cloud Computing Knowledge
  • Basic Programming Skills

Connect

we'd love to have your feedback on your experience so far