+- Softwarez.Info - Software's World! (https://softwarez.info)
+-- Forum: Library Zone (https://softwarez.info/Forum-Library-Zone)
+--- Forum: Video Tutorials (https://softwarez.info/Forum-Video-Tutorials)
+--- Thread: Snowflake - Build & Architect Data Pipelines Using Aws (/Thread-Snowflake-Build-Architect-Data-Pipelines-Using-Aws)
Snowflake - Build & Architect Data Pipelines Using Aws - OneDDL - 08-09-2024
![[Image: 101f68afcbefe2bbe66e1443ce3e69e1.jpeg]](https://i123.fastpic.org/big/2024/0808/e1/101f68afcbefe2bbe66e1443ce3e69e1.jpeg)
Free Download Snowflake - Build & Architect Data Pipelines Using Aws
Last updated 2/2023
MP4 | Video: h264, 1920x1080 | Audio: AAC, 44.1 KHz
Language: English | Size: 3.42 GB | Duration: 8h 48m
Data engineering and architecting pipelines using snowflake & AWS cloud
What you'll learn
Will learn everything needed for Snowpro Advanced Data engineering certification
Snowflake as a data-warehouse & automated pipelines within snowflake ecosystem
Use AWS Cloud with Snowflake as a data-warehouse
Integrating real time streaming data and data orchestration with Airflow and Snowflake
Requirements
Prior programming experience in Sql and python is a must .
Prior basic experience or understanding of cloud services like AWS is important
Description
Course Update as of Feb 2023 : This Course has been updated with Snowpark API which covers UDFs,Stored Procedures for ETL and also covers Machine Learning use-case deployments . This course will help you clear SnowPro Advanced CertificationsSnowflake is the next big thing and it is becoming a full blown data eco-system . With the level of scalability & efficiency in handling massive volumes of data and also with a number of new concepts in it ,this is the right time to wrap your head around Snowflake and have it in your toolkit . This course not only covers the core features of Snowflake but also teaches you how to deploy python/pyspark jobs in AWS Glue and Airflow that communicate with Snowflake , which is one of the most important aspects of building pipelines . Anyone who has a basic understanding of cloud and belong to one of the below backgrounds can benefit from this course :- Data Scientists / Analysts - Data Engineers / Software Developers - SQL Programmers or DBA's - Aspiring Data analysts and scientists who are learning SQL and Python This Course covers : What is Snowflake Most Crucial Aspects of Snowflake in a very practical manner Writing Python/Spark Jobs in AWS Glue Jobs for data transformationReal Time Streaming using Kafka and Snowflake Interacting with External Functions & use casesSecurity Features in Snowflake Prerequisites for this course are : Knowing SQL or at least some prior knowledge in writing queries Scripting in Python (or any language )Willingness to explore ,learn and put in the extra effort to succeed An active AWS Account & know-how of basic cloud fundamentals Important Note - You need to have an active AWS Account in order to perform tasks in sections related to Python and PySpark . For the rest of the course , a free trial snowflake account should suffice . Some Tips : Try to watch the videos at 1.2X speed Read the reference links and the official documentation of Snowflake as much as possible
Overview
Section 1: Introduction
Lecture 1 Course Roadmap
Lecture 2 Prerequisites and How to Success in this course
Lecture 3 Lecture 4 - Feedback and Learn More
Lecture 4 Clone Github Repo & PPT for the Course
Section 2: Introduction to Snowflake and AWS
Lecture 5 What is a data-warehouse ?
Lecture 6 Two Aspects of a Data Ecosystem
Lecture 7 Lab - Setup Snowflake Trial Account
Lecture 8 Snowflake Architecture
Lecture 9 Snowflake Object Heirarchy
Lecture 10 Snowflake - Virtual Warehouses
Lecture 11 Snowflake - Different Billing Components
Lecture 12 Snowflake - Track your consumption
Lecture 13 Snowflake- Resource Monitors
Section 3: Snowflake - Tables
Lecture 14 Introduction - Different Tables in Snowflake
Lecture 15 Lab - Create Tables in Snowflake
Lecture 16 Snowflake - Views , Materialized Views and Secure Views
Lecture 17 Lab - Create Views in Snowflake
Lecture 18 Lab - Create Secure Views in Snowflake
Lecture 19 More about Views in Snowflake
Section 4: Snowflake - Partitioning , Clustering and Performance Optimization
Lecture 20 Section Overview
Lecture 21 Introduction to partitions and clustering keys
Lecture 22 Lab - Micropartitions and Clustering keys
Lecture 23 Benefits of Micro-partitions and Clustering
Lecture 24 Understanding Clustering Depth and Cluster Overlap
Lecture 25 Lab - Selecting your clustering Keys
Lecture 26 Lab - Check Query Profile and history
Lecture 27 Lab - Query Processing and Caching
Lecture 28 Search Optimization Feature
Section 5: Snowflake - Data Loading/Ingestion and Extraction
Lecture 29 Section Overview
Lecture 30 Data Ingestion - Real World Use Cases
Lecture 31 Lab - Create an Integration Object to Connect Snowflake with AWS S3
Lecture 32 Lab - Ingest CSV from S3 to Snowflake
Lecture 33 Lab - Ingest JSON from S3 to Snowflake
Lecture 34 Introduction to Continuous Data Ingestion in Snowflake
Lecture 35 Lab - Create and implement Snowpipe
Lecture 36 Snowpipe - Billing Estimation and Key Considerations for Data Ingestion
Lecture 37 Lab - Extracting/Unload Data from Snowflake to S3
Section 6: Snowflake - Tasks and Query Scheduling
Lecture 38 Section Overview
Lecture 39 Introduction to Tasks
Lecture 40 Lab - Create Standalone and Dependent tree of tasks
Lecture 41 Lab - Billing and Query History for Tasks
Section 7: Snowflake - Streams and Change Data Capture
Lecture 42 Section Overview
Lecture 43 Introduction to Streams
Lecture 44 Lab - Implement Standard Streams
Lecture 45 Lab - Implement Append-Only Streams
Lecture 46 Lab - Streams in a Transaction
Lecture 47 Streams - Data Retention and Staleness
Lecture 48 Lab - Change Tracking using "Changes"
Lecture 49 Project Overview
Lecture 50 Lab - Create Streams - Project Solution Part-1
Lecture 51 Lab - Create Streams - Part-1 Continuation
Lecture 52 Lab - End to End Pipeline in Action
Section 8: Snowflake - User Defined Functions
Lecture 53 Introduction to User Defined Functions and UDF Types
Lecture 54 Lab - Write and implement a Scalar UDF
Lecture 55 Lab - Write Tabular UDF in SQL
Lecture 56 Lab - Implement Javascript UDFs
Lecture 57 What is Pushdown in UDF ?
Lecture 58 Lab - How can pushdown expose the underlying data ?
Lecture 59 Lab - Write Secure UDFs
Section 9: Snowflake - External Functions
Lecture 60 Section Overview
Lecture 61 Introduction to External Functions
Lecture 62 Lab - Write Deploy AWS Lambda Function
Lecture 63 Create IAM Role
Lecture 64 Lab - Create API Gateway
Lecture 65 Lab - Securing and Deploy API Gateway
Lecture 66 Lab - Create External Function in Snowflake
Section 10: Snowflake with Python,Spark and Airflow on AWS
Lecture 67 Section Overview
Lecture 68 Lab - Connect Python with Snowflake in your local machine
Lecture 69 Introduction to AWS Glue
Lecture 70 Lab - Deploy and execute python script to AWS Glue
Lecture 71 Lab - Parameterize your python script on AWS Glue
Lecture 72 Lab - Python Pandas with Snowflake on AWS Glue
Lecture 73 What is Pushdown in Spark 3.1 ?
Lecture 74 Lab - Deploy a Pyspark script using AWS Glue
Lecture 75 Lab - Setup Managed Airflow Cluster on AWS
Lecture 76 Lab - Configure Snowflake Connectivity in Airflow
Lecture 77 Lab - Deploy a PySpark Transformation job in AWS Glue
Lecture 78 Lab - Setup Airflow DAG
Section 11: Real Time Streaming with Kafka and Snowflake
Lecture 79 Section Overview
Lecture 80 Lab - Download the necessary JAR Files
Lecture 81 Lab - Setup Kafka in your local system
Lecture 82 Lab - Setup Kafka Snowflake Connector
Lecture 83 Lab - Setup Encryption Keys for Kafka-Snowflake Connectivity
Lecture 84 Lab - Streaming Data in Action
Section 12: Snowflake - Data Protection and Governance
Lecture 85 Section Overview
Lecture 86 What is TimeTravel and Failsafe in Snowflake ?
Lecture 87 Lab - Time Travel and Data Recovery
Lecture 88 Lab - Column Level Dynamic Data Masking
Lecture 89 What is Row Level Security ?
Lecture 90 Lab - Create and implement Row Level Access Policy
Lecture 91 More updates soon
Section 13: Strengthen your understanding - Bonus
Section 14: Snowpark - For Data Pipelines and Data Science
Lecture 92 Introduction-What is Snowpark?
Lecture 93 Lab - Getting Started with Snowpark
Lecture 94 Overview - UDFs and Store Procedures
Lecture 95 Lab-Deploy Python UDFs
Lecture 96 Lab-Deploy Stored Procedures for ETL Batch Processing
Lecture 97 Data Science - UseCase Overview and data preparation
Lecture 98 Lab-Deploy Model-Training Code for scikit-learn using Stored Procedures
Lecture 99 Lab-Deploy Model Serving/Prediction Serving Pipeline using UDFs
Lecture 100 More learning reference and Coupon Code
software engineers,aspiring data engineers or data analyst & data scientists,Also good for programmers and database administrators with experience in writing SQL queries
Homepage
Recommend Download Link Hight Speed | Please Say Thanks Keep Topic Live
No Password - Links are Interchangeable