Problem Solving Recipes Learned from Supporting Spark

Discover essential problem-solving techniques for Apache Spark in this 40-minute conference talk from Scala Days Copenhagen 2017. Explore common pitfalls encountered in big data applications, including OutOfMemoryErrors, NoSuchMethodErrors, and challenges with shuffle and partition management. Gain insights into less frequent but equally perplexing issues such as SQL broadcast configurations, speculation struggles, stream recovery failures due to RDD joins, and S3 file reading hangs. Learn valuable tips and tricks from Lightbend experts to avoid these mistakes, potentially saving hours or days of debugging time. Whether you're an experienced Spark developer or a newcomer to the field, acquire practical knowledge to enhance your big data application development and troubleshooting skills.

Problem Solving Recipes Learned from Supporting Spark by Stavros Kontopoulos & Justin Pihony