Netdata: Open Source Distributed Observability Pipeline

Explore an in-depth conference talk on Netdata, an open-source distributed observability pipeline, presented by Costa Tsaousis at Conf42 Observability 2024. Dive into the challenges of modern observability and learn how Netdata addresses them through its decentralized design, real-time monitoring, and AI-powered analytics. Discover the tool's approach to high-fidelity data collection, automated dashboards, and efficient log exploration. Gain insights into Netdata's lightweight architecture, its position in the CNCF landscape, and how it compares to other platforms like Prometheus. Understand the implementation of machine learning for anomaly detection and the integration of various data types beyond traditional metrics, logs, and traces. Conclude with a look at Netdata's monetization strategy and its vision for the future of observability.

intro
preamble
about netdata
why another tool?
the current state of observability
observability generations
attributes of the ideal observability solution...
goal: monitor everything!
goal: real-time per second & low latency!
goal: powerful fully automated dynamic dashboards!
goal: ai-powered correlations!
netdata
the design of observability keeps fidelity low
decentralized design for high fidelity
common concerns about decentralized designs
netdata on an empty vm
distributed metrics pipeline
- standalone
- distributed monitoring
- centralization points
- hybrid cloud
netdata vs prometheus
netdata is the most lightweight platform!
netdata in cncf
we have a lot in common
nidl, the model for rapid deployment
nidl: how it looks?
nidl: the result
mission accomplished!
slice and dice from the ui
a netdata chart
info button: help about this chart
a netdata chart - controls
a netdata chart - anomaly rate per node
dicing any chart, without queries
info ribbon: missing data collections
mission accomplished!
ai for observability is tricky
what does netdata do with machine learning?
netdata's scoring engine
a netdata dashboard
anomaly advisor
mission accomplished!
systemd-journald
netdata systemd-journal logs explorer
systemd-journald: it is slow to query
systemd-journald: it lacks integration
mission accomplished!
challenge
netdata functions
example: network connections explorer
mission accomplished!
monetization through saas
thank you!