Crawling the Web with Python and Scrapy

Are you trying to gather high-quality data from specific websites, and wondered how you could extract this data programmatically. In this course you will gain the skills and knowledge on how to use Scrapy with Python, to programmatically crawl and scrape data from any website.

Have you ever spent hours trying to gather high-quality data from specific websites, and wondered how you could extract this data programmatically and use it within your own applications? In this course, Crawling the Web with Python 3 and Scrapy 2, you will gain the ability to write spiders that can extract data from the web, using Python and Visual Studio Code, through an advanced yet easy-to-use framework called Scrapy. First, you will learn what scraping and crawling are, and explore all its implications. Next, you will discover how to scaffold a Scrapy project and write spiders. Finally, you will explore how to influence how spiders crawl websites and extract data in different formats. When you are finished with this course, you will have the skills and knowledge on how to use Scrapy with Python, to programmatically crawl and scrape data from any website.

• Course Overview 1min
• Extracting Data from the Web – Core Concepts 27mins
• Scaffolding and Running Your First Scrapy Web Crawler Project 16mins
• Achieving Common Spider Behaviors Using Built-in Classes 24mins
• Influencing Scrapy Crawling 15mins
• Scrapy Outcome and Data Export 5mins