Last Week in a Byte on Delta Lake | 2023-03-28

You can watch or read the latest #DeltaLake news a week late (2023-03-28 edition)!

Recent Releases

We’re excited about the Delta Lake 2.3.0 Preview on Apache Spark 3.3 with key features we’re previewing, including:

• Zero-copy convert to Delta from Iceberg tables
• Support SHALLOW CLONE for Delta, Parquet, and Iceberg tables
• Support idempotent writes for DML operations
• Support reading Change Data Feed (CDF) in SQL queries
• Support reading Delta tables with deletion vectors, and much much more!

For more information, check out Release Delta Lake 2.3.0 · delta-io/delta (github.com)

Recent Events

Matthew Powers, CFA and Miguel Angel Diaz Rodriguez , MLOps Director at AB InBev presented Model Serving with MLflow and Delta Lake main features at Apache Spark Bogata meetup 

We recently published Delta Lake D3L2 vidcast Implementing a Data Lakehouse for Improved Data Science and Analytics at T-Mobile with Robert Thompson and Geoff Freeman , Members of Technical Staff at T-Mobile .

Upcoming Events

If you're in the Seattle area in April, there are two #pydata events:

On April 25th, there is a Seattle Spark+AI Meetup featuring

• Introduction to Spark Connect with Chengyin Eng and @ Hyukjin Kwon
• Spark/Ray/MLflow/Delta Lake panel with Eloisa E. (PyData), Jules Damji ( Anyscale ), and Denny Lee

On April 26th-28th, there is the PyData Seattle 2023 conference featuring the following sessions:

•  Building Reliable, Open Lakehouses with Delta Lake with Jim Hibbard and Nick Karpov
• MLOps Deployment Patterns with Delta Lake and MLflow with Mary Grace Moesta

Latest community blogs

• Jacek Laskowski's is adding more to his Internals of Delta Lake book, calling out #DeltaLake 2.2 metadata operations like LIMIT

• Nick Karpov published Z-Order: Visualization and Implementation, exploring the Delta Lake Spark connector’s Z-Order command through visualization and implementation. #zorder #apachespark
• Denny Lee 's first blog as part of the “Ask Delta?” blog series is Why does altering a Delta Lake table schema not show up in the Spark DataFrame? #schema #apachespark
• Matthew Powers, CFA published How to Convert from CSV to Delta Lake #csv
• Jim Hibbard published Running ML Workflows with Delta Lake and Ray (Part 1) #rayml

Connect with us!

Want to learn more about Delta Lake and chat with other users and contributors? Join us at delta.io, Slack, LinkedIn, and GitHub.