Let's Play with Charmed Spark - Apache Spark on Kubernetes

Explore the world of Apache Spark and its deployment on Kubernetes using Charmed Spark in this informative conference talk from Ubuntu Summit 2023. Learn how to set up and run Spark workloads efficiently using tools supported by Canonical. Follow along as the speaker demonstrates deploying a Kubernetes cluster with MicroK8s, configuring roles and permissions with Spark Client snap, and utilizing spark-shell and pyspark utilities for interactive Spark usage in Scala or Python. Discover how to submit regular jobs, monitor their status through the Spark history server, and integrate this Spark solution with other Data Platform products like Kafka. Gain insights into computing metrics over data streams produced by Kafka using Spark's streaming engine. Perfect for data scientists, engineers, and administrators looking to simplify their Spark workflows on Kubernetes.

Let's play with Charmed Spark