Data Engineering On Microsoft Azure

[BaDshaH]

Published 4/2024
MP4 | Video: h264, 1920x1080 | Audio: AAC, 44.1 KHz
Language: English | Size: 3.44 GB | Duration: 6h 24m

DP 203 - Azure Data Engineer Associate

What you'll learn
Implement a partition strategy
Design and implement the data exploration layer
Ingest and transform data
Develop a batch processing solution
Develop a stream processing solution
Manage batches and pipelines
Implement data security
Monitor data storage and data processing
Optimize and troubleshoot data storage and data processing

Requirements
Foundational Knowledge of Azure

Description
In this course, you will learn how to implement and manage data engineering workloads on Microsoft Azure, using Azure services such as Azure Synapse Analytics, Azure Data Lake Storage Gen2, Azure Stream Analytics, Azure Databricks, and others. The course focuses on common data engineering tasks such as orchestrating data transfer and transformation pipelines, working with data files in a data lake, creating and loading relational data warehouses, capturing and aggregating streams of real-time data, and tracking data assets and lineage. You can become a data professional, a data architect, or a business intelligence professional by learning about data engineering and building analytical solutions using data platform technologies that exist on Microsoft Azure. This course will give you a flavor of end-to-end processing of big data in Azure.As a candidate for this certification, you should have subject matter expertise in integrating, transforming, and consolidating data from various structured, unstructured, and streaming data systems into a suitable schema for building analytics solutions.As an Azure data engineer, you help stakeholders understand the data through exploration, and build and maintain secure and compliant data processing pipelines by using different tools and techniques. You use various Azure data services and frameworks to store and produce cleansed and enhanced datasets for analysis.

Overview
Section 1: Microsoft Azure Data Engineering Introduction
Lecture 1 Introduction
Section 2: Get started with data engineering on Azure
Lecture 2 Introduction to data engineering on Azure
Lecture 3 Introduction to Azure Data Lake Storage Gen2
Lecture 4 Introduction to Azure Synapse Analytics
Lecture 5 Lab - Explore Azure Synapse Analytics
Section 3: Build data analytics solutions using Azure Synapse Analytics serverless SQL pool
Lecture 6 Use a serverless SQL pool to query files in a data lake
Lecture 7 Use a serverless SQL pool to transform data
Lecture 8 Lab - Transform files using a serverless SQL pool
Lecture 9 Create a lake database
Section 4: Perform data engineering with Azure Synapse Apache Spark Pools
Lecture 10 Analyze data with Apache Spark in Azure Synapse Analytics
Lecture 11 Transform data with Apache Spark in Azure Synapse Analytics
Lecture 12 Lab - Transform data using Spark in Synapse Analytics
Lecture 13 Use Delta Lake in Azure Synapse Analytics
Lecture 14 Lab - Use Delta Lake with Spark in Azure Synapse Analytics
Section 5: Work with data warehouses using Azure Synapse Analytics
Lecture 15 Analyze data in a relational data warehouse
Lecture 16 Load data into a relational data warehouse
Lecture 17 Lab - Load Data into a Relational Data Warehouse
Section 6: Transfer and transform data with Azure Synapse Analytics Pipelines
Lecture 18 Build a data pipeline in Azure Synapse Analytics
Lecture 19 Lab - Build a data pipeline in Azure Synapse Analytics
Lecture 20 Use Spark Notebooks in an Azure Synapse Pipeline
Lecture 21 Lab - Use an Apache Spark notebook in a pipeline
Section 7: Hybrid transactional and analytical processing Solutions using Synapse Analytics
Lecture 22 Plan hybrid transactional and analytical processing
Lecture 23 Implement Azure Synapse Link with Azure Cosmos DB
Lecture 24 Lab - Use Azure Synapse Link for Azure Cosmos DB
Lecture 25 Implement Azure Synapse Link for SQL
Section 8: Implement a data streaming solution with Azure Stream Analytics
Lecture 26 Get started with Azure Stream Analytics
Lecture 27 Streaming data using Azure Stream Analytics and Azure Synapse Analytics
Lecture 28 Lab - Ingest data with Azure Stream Analytics and Azure Synapse Analytics
Lecture 29 Visualize real-time data with Azure Stream Analytics and Power BI
Section 9: Govern data across an enterprise
Lecture 30 Introduction to Microsoft Purview
Lecture 31 Integrate Microsoft Purview and Azure Synapse Analytics
Lecture 32 Lab - Use Microsoft Purview with Azure Synapse Analytics
Section 10: Data engineering with Azure Databricks
Lecture 33 Explore Azure Databricks
Lecture 34 Use Apache Spark in Azure Databricks
Lecture 35 Lab - Use Spark in Azure Databricks
Lecture 36 Run Azure Databricks notebooks in Azure Data Factory
Section 11: Conclusion
Lecture 37 Redemption of Badges
Data Engineers: Professionals who focus on preparing "big data" for analytical or operational uses. These individuals are responsible for designing, building, and maintaining the architecture (such as databases and large-scale processing systems) for data ingestion, processing, and analytics.,Data Architects: These are professionals who design the blueprint for managing data across the organization. They work on designing data solutions that utilize Azure services effectively to meet both the technical and business requirements.,Data Professionals: This broad category includes anyone working with data in a technical capacity and looking to leverage Azure's data services for their data solutions. This could include database administrators, data analysts, and software developers with a focus on data.,IT Professionals: IT professionals who are not necessarily data specialists but are looking to expand their skills into the data engineering space can benefit from this course. This includes system administrators, software developers, and IT managers who need to understand how data solutions are designed and implemented on Azure.,Students and Recent Graduates: Those who are studying in fields related to computer science, data science, information technology, or similar areas and are looking to enter the workforce with a strong set of skills in cloud-based data solutions.

Homepage

[To see links please register or login]

[To see links please register or login]