Data Processing at Scale with Knative and Benthos

Explore a conference talk on leveraging Knative and Benthos for large-scale data processing. Learn how Knative serving provides push-based autoscaling for real-time HTTP/gRPC requests, and discover solutions for event and batch processing in enterprise scenarios. Understand the challenges of processing batch data from databases and CSV files, and see how Benthos, a stateless data streaming engine, addresses these issues with transaction-based resiliency and backpressure. Dive into Benthos' capabilities, including at-least-once delivery guarantees and data transformations using a domain-specific language. Watch as the speakers demonstrate how to combine Knative and Benthos to efficiently process data at scale, showcasing practical applications for enterprise use cases.

Data Processing at Scale with Knative and Benthos - Mihai Todor & Murugappan Sevugan Chetty, Box