YoVDO

Use Spark from Anywhere - A Spark Client in Scala Powered by Spark Connect

Offered By: Scala Days Conferences via YouTube

Tags

Scala Days Courses Scala Courses

Course Description

Overview

Save Big on Coursera Plus. 7,000+ courses at $160 off. Limited Time Only!
Explore the evolution of Apache Spark and its new client-server architecture in this Scala Days 2023 Seattle conference talk. Discover how Spark Connect enables remote connectivity to Spark clusters using the DataFrame API and unresolved logical plans. Learn about the benefits of separating client and server, allowing Spark to be leveraged from various environments including modern data applications, IDEs, notebooks, and programming languages. Gain insights into the development of the Scala client for Spark Connect, its integration with the server, and efforts to solve application isolation challenges. Dive deep into Spark Connect's architecture and understand how the community can extend it for new programming languages and frameworks. Presented by Denny Lee, a Databricks Developer Advocate with expertise in distributed systems and data sciences, and Ginger Holt, a Senior Staff Data Scientist at Databricks specializing in forecasting and predictive models.

Syllabus

Denny Lee & Ginger Holt - Use Spark from Anywhere | Scala Days 2023 Seattle


Taught by

Scala Days Conferences

Related Courses

Teaching Domain Specific Languages in Scala
Scala Days Conferences via YouTube
Why Dolly Is Just the Beginning for Open LLM Models
Scala Days Conferences via YouTube
Building Billion Node Graphs for Machine Learning
Scala Days Conferences via YouTube
How Does Incremental Compilation Work with Scala 3
Scala Days Conferences via YouTube
AI Assisted Development
Scala Days Conferences via YouTube