Context Aware Traffic Routing for Stateful Services Using Envoy

Explore context-aware traffic routing for stateful services using Envoy in this conference talk by Renana Yacobi and Piaw Na from Niantic. Learn how to implement invisible service infrastructure for application APIs, addressing challenges related to sensitive information like PII and company IP. Discover techniques for routing decisions based on message content, including the use of Envoy custom filters and partial message parsing. Gain insights into retrofitting protocols, performing smarter routing, and improving performance through client-server mapping coordination. Examine load test setups, resource utilization, and the benefits of partial message deserialization for large messages.

Thank you to our session recording sponsor!
Context aware routing We can route messages based on the context of the message Context is provided by: . URL routing (usually used by websites) • Request Header leg used by load balancers • Side channel (usually used by apps) . As part of the message content
Use cases • Retrofitting protocol based on content without application being aware • Use existing content to perform smarter routing
Custom filter overview
Load test setup Test Message size average OS KB
Protobuf Wireformat
Things of Note • Envelope can be retrofitted ., envelope does not have to be . The unique string can be changed to indicate backwards Oncompatible changes . More than one kind of address can be used for routing purposes
Things of Note • Envelope can be retrofitted ., envelope does not have to be . The unique string can be changed to indicate backwards Incompatible changes . More than one kind of address can be used for routing purposes
Load test setup (V2) Test
Test 1 Resource Utilization (V2)
Stateful message routing
Stateful services message routing
Proposed solution
Client server mapping coordinator
Conclusions • Shifting the client to server mapping can help with improving • Partial message deserialization can help when dealing with large message