Data Engineering with Databricks
Data professionals from all walks of life will benefit from this comprehensive introduction to the components of the Databricks Lakehouse Platform that directly support putting ETL pipelines into production. You will leverage SQL and Python to define and schedule pipelines that incrementally process new data from a variety of data sources to power analytic applications and dashboards in the Lakehouse. This course offers hands-on instruction in Databricks Data Science & Engineering Workspace, Databricks SQL, Delta Live Tables, Databricks Repos, Databricks Task Orchestration, and the Unity Catalog.
Languages Available: English | 日本語 | Português BR | 한국어
- Before attempting this course, please ensure that you meet these prerequisites. If you don’t, you might have trouble following along with course content.
Note that this course provides context for how to perform the actions within it using both Spark SQL and PySpark. If you are interested in one over the other, feel free to focus on that content (and skip the videos that are not relevant to you).
If you plan to follow along in Spark SQL, here are your pre-requisites:
- * Beginner familiarity with cloud computing concepts (virtual machines, object storage, etc.)
- * Production experience working with data warehouses and data lakes
- * Intermediate experience with basic SQL concepts (select, filter, groupby, join, etc)
If you plan on following along in both Spark SQL and PySpark, please see these additional pre-requisites:
* Beginner programming experience with Python (syntax, conditions, loops, functions)
* Beginner programming experience with the Spark DataFrame API (PySpark)
* Configure DataFrameReader and DataFrameWriter to read and write data (PySpark)
* Express query transformations using DataFrame methods and Column expressions (PySpark)
* Navigate the Spark documentation to identify built-in functions for various transformations and data types (PySpark)
Outline
Day 1:
Module 1: Get Started with Databricks Data Science and Data Engineering Workspace
Module 2: Transform Data with Spark
Module 3: Manage Data with Delta Lake
Day 2:
Module 4: Build Data Pipelines with Delta Live Tables
Module 5: Deploy Workloads with Databricks Workflows
Module 6: Manage Data Access with Unity Catalog
Upcoming Public Classes
Date | Time | Language | Price |
---|---|---|---|
Jul 15 - 16 | 09 AM - 05 PM (Australia/Sydney) | English | $1500.00 |
Jul 22 - 23 | 08 AM - 04 PM (America/Chicago) | English | $1500.00 |
Jul 24 - 25 | 09 AM - 05 PM (Europe/London) | English | $1500.00 |
Jul 29 - Aug 01 | 11 AM - 03 PM (Asia/Singapore) | English | $1500.00 |
Jul 29 - 30 | 09 AM - 05 PM (Europe/Paris) | English | $1500.00 |
Jul 29 - Aug 01 | 08 AM - 12 PM (America/Chicago) | Spanish | $1500.00 |
Aug 05 - 06 | 09 AM - 05 PM (Europe/London) | English | $1500.00 |
Aug 05 - 06 | 09 AM - 05 PM (America/New_York) | English | $1500.00 |
Aug 12 - 13 | 10 AM - 06 PM (Asia/Singapore) | English | $1500.00 |
Aug 12 - 13 | 08 AM - 04 PM (America/Chicago) | English | $1500.00 |
Aug 19 - 20 | 09 AM - 05 PM (Asia/Kolkata) | English | $1500.00 |
Aug 19 - 20 | 08 AM - 04 PM (America/Los_Angeles) | English | $1500.00 |
Aug 26 - 27 | 09 AM - 05 PM (Australia/Sydney) | English | $1500.00 |
Aug 26 - 27 | 09 AM - 05 PM (America/New_York) | English | $1500.00 |
Aug 28 - 29 | 09 AM - 05 PM (Europe/Paris) | English | $1500.00 |
Sep 02 - 03 | 09 AM - 05 PM (Europe/London) | English | $1500.00 |
Sep 10 - 11 | 09 AM - 05 PM (Europe/Paris) | English | $1500.00 |
Sep 12 - 13 | 10 AM - 06 PM (America/New_York) | English | $1500.00 |
Sep 24 - 25 | 09 AM - 05 PM (Europe/Paris) | English | $1500.00 |
Sep 30 - Oct 01 | 09 AM - 05 PM (Europe/London) | English | $1500.00 |
Oct 07 - 08 | 09 AM - 05 PM (America/Los_Angeles) | English | $1500.00 |
Oct 16 - 17 | 09 AM - 05 PM (Europe/London) | English | $1500.00 |
Oct 24 - 25 | 09 AM - 05 PM (Europe/Paris) | English | $1500.00 |
Oct 28 - 29 | 09 AM - 05 PM (Europe/London) | English | $1500.00 |
Nov 07 - 08 | 09 AM - 05 PM (America/New_York) | English | $1500.00 |
Nov 18 - 19 | 09 AM - 05 PM (Europe/Paris) | English | $1500.00 |
Nov 25 - 26 | 09 AM - 05 PM (Europe/London) | English | $1500.00 |
Dec 02 - 03 | 09 AM - 05 PM (Europe/Paris) | English | $1500.00 |
Dec 03 - 04 | 10 AM - 06 PM (America/New_York) | English | $1500.00 |
Dec 16 - 17 | 09 AM - 05 PM (Europe/Paris) | English | $1500.00 |
Public Class Registration
If your company has purchased success credits or has a learning subscription, please fill out the Training Request form. Otherwise, you can register below.
Private Class Request
If your company is interested in private training, please submit a request.
Registration options
Databricks has a delivery method for wherever you are on your learning journey
Self-Paced
Custom-fit learning paths for data, analytics, and AI roles and career paths through on-demand videos
Register nowInstructor-Led
Public and private courses taught by expert instructors across half-day to two-day courses
Register nowBlended Learning
Self-paced and weekly instructor-led sessions for every style of learner to optimize course completion and knowledge retention. Go to Subscriptions Catalog tab to purchase
Purchase nowSkills@Scale
Comprehensive training offering for large scale customers that includes learning elements for every style of learning. Inquire with your account executive for details
Upcoming Public Classes
Career Workshop/
March 20