Web Scraping in Python 101

Learn the fundamentals of web scraping using Python in this conference talk. Explore various Python libraries for web scraping, compare open-source and proprietary alternatives, and discover when to use specific tools like Scrapy. Gain insights into real-world applications, advantages over APIs, and essential techniques for extracting data from websites. Understand the workings of different scraping libraries, their strengths, and limitations. Dive into practical examples, including building a custom scraper with Scrapy, and learn best practices for handling exceptions, rate limiting, and data verification. Suitable for both beginners and experienced programmers looking to enhance their web scraping skills.

Intro
About Me
My Experience
Agenda
What is web scraping
Web scraping vs API
Web scraping use cases
Advantages of web scraping
Essential parts of web scraping
Beautiful Soup
HTTP Libraries
Paging Libraries
BeautifulSock
Lxml
scraping needs
What is Scrap
When to use Scrap
Scrap workflow
Project name
Items
Extracting
Full Spider
Scrap Space
Scrap Data
Field Exports
When not to use scraping
What should you use
What do I prefer
YouTubeDL
Conclusion
Exception Handling
Verification
Rate Limit
Python3 Support