Advanced Stream Processing on the Edge

Explore advanced stream processing techniques for edge computing in this 39-minute conference talk from linux.conf.au 2020. Delve into the evolution of logging mechanisms for distributed systems and connected embedded devices. Learn about real-time data analysis, stream processing on the edge, and how to query data using aggregation functions, windowing, and grouping results while it's still in motion and in-memory. Discover the advantages of processing data on edge services or devices, including anomaly detection and distributed machine learning using open-source software. Gain insights into data pipelines, JSON handling, time series forecasting, and performance optimization for edge computing scenarios.

Introduction
Logging
Challenges
Data Source
Data Structure
Flu Embed
Data Pipeline
Simple Messages
Metadata
Record
JSON
Daemon Set
Stream Processing
How Stream Processing Works
Goal of Stream Processing
Stream Processing in Fluent Bit
Stream Processing in Data Pipeline
Time Series Forecasting
Internals
Synchronous Operations
Helpers
Performance