Professional Data - Wrestling Techniques Using Elasticsearch's Aggregation Framework

Explore powerful data analysis techniques using Elasticsearch's Aggregation Framework in this 45-minute conference talk from GOTO Amsterdam 2015. Discover how to extract valuable insights from large datasets without the need for expensive data scientists. Learn through practical examples including UK housing data, crime statistics, Enron emails, and MovieLens recommendations. Understand the importance of representative sampling and how to combine search and analytics effectively. Dive into real-world applications such as Amazon marketplace reviews and UK car roadworthiness tests. Gain insights into entity indexing, derived attributes, and data visualization techniques. Master the art of wrestling with big data using Elasticsearch's out-of-the-box algorithms and optimize your data analysis workflow for real-time results.

Introduction
Search interface 20
Analytics interface of today
Optimised for real time
Worked examples
UK Housing data: percentiles
UK Housing data: terms
Geo as a common link between datasets: housing crime
Connected data: Enron emails
Recommendations: MovieLens data
Random samples should hold no surprises
Non random sample: people who liked Talladega nights
Problem: avoid analysis of poorly focused sets
How do we get a smaller, representative sample of users?
Putting search and analytics together..
Amazon marketplace reviews
Anatomy of an entity indexing groovy script
Drilling down into seller #187's fanboys
UK car roadworthiness test: raw data
Derived car attributes
Miles driven vs number of days for fix
In summary
Questions?