Onyx - Distributed Workflows for Dynamic Systems

Explore a conference talk on Onyx, a distributed computation system written in Clojure, that addresses common pain points in complex distributed activity processing. Dive into the challenges of workflow specification, stateful lifecycle management, and developer testing in contemporary frameworks like Storm and Cascading. Learn how Onyx leverages immutability, persistent data structures, queues, and transactions to simplify and change processing workflows. Discover how this innovative system adapts to modern hardware advancements, utilizing fast networks and SSDs to remove incidental complexity. Gain new perspectives on tackling increasingly complex problem spaces through concepts such as data representation, functions, transducers, positional representation, execution context, extensibility model, runtime discovery, and task state API. Understand the architecture overview and performance considerations of Onyx, presented by software engineering consultant Michael Drogalis at the Strange Loop Conference.

Intro
What is Onyx?
Onyx Goals
The Problem
Cascading: A Solution?
Storm: A Solution?
Cascalog: A Solution?
Decomposing the Monolith
Guiding Example
Data Representation
Functions
Did Somebody Say Transducers?!
Positional Representation
Execution Context
Extensibility Model
Runtime Discovery
Task State API
Architecture Overview
Wait! That's Slow!
Questions?