Following Google or Don't Follow the Followers, Follow the Leader

Explore the evolution of data management and technology in this thought-provoking conference talk from GOTO Chicago 2014. Delve into the reasons for following Google's technological lead, examining their unprecedented scale and ability to attract top talent. Analyze the challenges of scale, speed, persistence, and context in modern data management. Critically examine popular trends like Hadoop and "big data," questioning their effectiveness compared to more established technologies. Gain insights into the lifecycle of data, the importance of understanding data structure and semantics, and the potential future of persistence layer processing. Learn about the historical development of information storage, from scrolls to digital databases, and how it impacts current technological choices. Investigate the shift from top-down taxonomies to more flexible, bottom-up approaches in organizing information. Discover how relational databases have evolved to handle increasing scale and the emergence of new architectures like sharding and stateless designs. Evaluate the strengths and weaknesses of various database technologies, including NoSQL and key-value stores, in the context of modern data challenges. Reflect on the importance of query processing, optimization, and data modeling in creating effective data management solutions.

Intro
Early history
Scrolls
Clay vs Paper
Codex
Medium encyclopedias
Movable type
Perfect copies
Metadata
Sharing knowledge
Taxonomy
Species are not fixed
Victorian Era
Industrial Era
Topdown Taxonomy
Why did this happen
Problems of pragmatism
Scale changes
Top down vs bottom up
Scale changes architectures
Big data
Relational databases
Moores law
Clientserver
Connection pooling
Stateless architectures
Database scalability
Sharding
Design for failure
You can put this new thing in
So what have we done
Query processing
Big mainframes
Data warehousing
Cubing architectures
Database architectures
No sequel
Query optimization
Data modeling
Performance curve
Key value store
Query strings
Hadoop
Microsoft
Model building
Unstructured data
Scientific computing
Query problems
Small analytics
Hadoop and no sequel
Big conclusion
F1 evolution