YoVDO

Apache Spark Essential Training

Offered By: LinkedIn Learning

Tags

Apache Spark Courses Big Data Courses Data Analysis Courses Machine Learning Courses Spark Streaming Courses Spark SQL Courses MLlib Courses

Course Description

Overview

Learn how to make Apache Spark work with other Big Data technologies and put together an end-to-end project that can solve a real-world business problem.

Syllabus

Introduction
  • Driving big data engineering with Apache Spark
  • Course prerequisites
  • Setting up the exercise files
1. Data Engineering Concepts
  • What is data engineering?
  • Data engineering vs. data analytics vs. data science
  • Data engineering functions
  • Batch vs. real-time processing
  • Data engineering with Spark
2. Spark Capabilities for ETL
  • Spark architecture review
  • Parallel processing with Spark
  • Spark execution plan
  • Stateful stream processing
  • Spark analytics and ML
3. Batch Processing Pipelines
  • Batch processing use case: Problem statement
  • Batch processing use case: Design
  • Setting up the local DB
  • Uploading stock to a central store
  • Aggregating stock across warehouses
4. Real-Time Processing Pipelines
  • Real-time use case: Problem
  • Real-time use case: Design
  • Generating a visits data stream
  • Building a website analytics job
  • Executing the real-time pipeline
5. Data Engineering with Spark: Best Practices
  • Batch vs. real-time options
  • Scaling extraction and loading operations
  • Scaling processing operations
  • Building resiliency
6. End-to-End Exercise Project
  • Project exercise requirements
  • Solution design
  • Extracting long last actions
  • Building a scorecard
Conclusion
  • More about Apache Spark

Taught by

Ben Sullins

Related Courses

Curso Completo de Spark con Databricks (Big Data)
Coursera Project Network via Coursera
Machine Learning con Spark (MLlib) en Databricks
Coursera Project Network via Coursera
ML y Big Data con PySpark para la retención de clientes
Coursera Project Network via Coursera
Machine Learning y Regresión con PySpark. Guía paso a paso
Coursera Project Network via Coursera
Taming Big Data with Apache Spark and Python - Hands On!
Udemy