Web Scraping with Python

Offered By: LinkedIn Learning

Course Description

Overview

This course dives into the process, strategies, and best practices of web scraping. Learn how to use the Python framework, Scrapy, to practice key techniques.

Syllabus

Introduction

How to learn to stop worrying and love the bot
What you should know

1. Basic Web Scraping

What is web scraping?
How the internet works: A brief summary
Hello world with Scrapy
Challenge: Scraping all data on a page
Solution: Scraping all data on a page

2. Learning to Crawl

Crawling a website
Recording data
Scrapy settings file
Structuring your scrapers for extensibility/reusability
Challenge: Scraping news sites
Solution: Scraping news sites

3. Advanced Techniques

Submitting a form
Finding and using hidden APIs
Sitemaps and robots.txt
Challenge: Using CNN's sitemap
Solution: Using CNN's sitemap

4. Acting Human

Logging in
Browser automation with Selenium
Interacting with a page

Conclusion

Next steps

Taught by

Ryan Mitchell

Related Courses

Web and Mobile Testing with Selenium
University of Minnesota via Coursera Create Your First Automation Script Using Selenium and Java
Coursera Project Network via Coursera Automate an e-commerce web application using Selenium & Java
Coursera Project Network via Coursera Advanced TestNG Framework and Integration with Selenium
Coursera Project Network via Coursera Behavior Driven Development with Selenium and Cucumber
Coursera Project Network via Coursera