Lychee: Writing a Link Checker in a Weekend (Plus Two Short Years)

Explore the journey of developing Lychee, an asynchronous Rust link checker, in this 50-minute conference talk from RustLab. Delve into the unexpected complexities of creating a seemingly simple network tool, including challenges with rate limiting, status codes, URL parsing, timeouts, and client-side rendering. Gain insights into open-source project funding, maintenance, and the essential skills required for development. Discover the intricacies of link checking, including handling various link types, edge cases, and bot detection. Learn about the tool's performance, its role in critical infrastructure, and its connection to Mozilla.

Introduction
Why I started this project
The name Lychee
The author of curl
Why Rust
What is a link
Valid links
Email addresses
Broken links
DOI links
Edge cases
Bot detection
Is it fast
Critical infrastructure
QA
Mozilla