Reordering 59 Million NYT Publishing Assets Using Go and BadgerDB
Offered By: Gopher Academy via YouTube
Course Description
Overview
Explore the process of reordering 59 million New York Times publishing assets using Go and BadgerDB in this GopherCon 2020 talk. Discover how to chronologically and topologically organize every version of NYT articles since 1851, along with related assets like images and tags. Learn about encoding assets, inserting them into databases, resolving missing references, and removing duplicates. Gain insights into Go development philosophy, BadgerDB performance optimization, and practical techniques such as string enumeration, timestamp handling, and error management in goroutines.
Syllabus
Intro
My Background
T Publishing Assets
Encode Assets and Insert Into Database
Find and Resolve Missing References
Find and Remove Duplicates
T Development Philosophy
Fast Performance (v 1.6.1)
I'm New to Go
T Badger Wrapper
T Batch DB Wrapper
TEncoding Strings via Enumeration
T Marshal - Read
T Timestamps as Keys
Refresher: Endianness
Endianness - Sorting
T Unique Timestamp
TUnique Timestamp Cache
T Panic Error Handling - Goroutines
Taught by
Gopher Academy
Related Courses
A Holistic Go ExperienceGopher Academy via YouTube Safe, Fast, and Easy - Building a Plugin System with WebAssembly
Gopher Academy via YouTube A Journey Through Integration Testing with Go - What Could Go Wrong?
Gopher Academy via YouTube Kraken Wrangling for Dummies - Go Module Dependency Management at Scale
Gopher Academy via YouTube Go, the Data Engineer's Missing Tool?
Gopher Academy via YouTube