Use Spark from Anywhere - A Spark Client in Scala Powered by Spark Connect
Offered By: Scala Days Conferences via YouTube
Course Description
Overview
Explore the evolution of Apache Spark and its new client-server architecture in this Scala Days 2023 Seattle conference talk. Discover how Spark Connect enables remote connectivity to Spark clusters using the DataFrame API and unresolved logical plans. Learn about the benefits of separating client and server, allowing Spark to be leveraged from various environments including modern data applications, IDEs, notebooks, and programming languages. Gain insights into the development of the Scala client for Spark Connect, its integration with the server, and efforts to solve application isolation challenges. Dive deep into Spark Connect's architecture and understand how the community can extend it for new programming languages and frameworks. Presented by Denny Lee, a Databricks Developer Advocate with expertise in distributed systems and data sciences, and Ginger Holt, a Senior Staff Data Scientist at Databricks specializing in forecasting and predictive models.
Syllabus
Denny Lee & Ginger Holt - Use Spark from Anywhere | Scala Days 2023 Seattle
Taught by
Scala Days Conferences
Related Courses
Teaching Domain Specific Languages in ScalaScala Days Conferences via YouTube Why Dolly Is Just the Beginning for Open LLM Models
Scala Days Conferences via YouTube Building Billion Node Graphs for Machine Learning
Scala Days Conferences via YouTube How Does Incremental Compilation Work with Scala 3
Scala Days Conferences via YouTube AI Assisted Development
Scala Days Conferences via YouTube