Modern Internet-Scale Network Reconnaissance - Underflow

Explore modern techniques for large-scale network reconnaissance in this BSidesLV conference talk. Delve into the challenges of current recon methods and learn how to build a platform for local data processing. Discover new tools and approaches for downloading and crunching massive datasets, including MTBL databases and JSON line formats. Examine the capabilities of various platforms and storage solutions. Investigate advanced topics such as Certificate Transparency logs, domain fronting across multiple cloud providers, and asset discovery during mergers and acquisitions. Gain insights into real-time CT monitoring, racing for first setup, and winning WordPress installations. Conclude with a roadmap for future developments and a live demonstration of cutting-edge reconnaissance techniques.

Introduction
Shiny New Recon Tools!
Problem Space
Current Challenges
Moving to Local Data
Build a Platform
Downloading Data
Crunching Data
Server Specifications
MTBL Databases
Convert Censys.io to MTBL
JSON Line Format
Storage Usage by Source
Platform Capabilities
Next Steps
Certificate Transparency Logs
Extended Validation in Chrome
Lets Encrypt + CT
Real-time CT Monitoring
Racing to First Setup
Winning a WordPress
Cloud Discovery: Azure DNS
Domain Fronting: Azure
Domain Fronting: Cloudfront
Domain Fronting: Fastly
Discovery: M&A
Discovery: Full Asset List
Summary
Roadmap
Demo Time!