Optimization for Number of Goroutines Using Feedback Control

Explore an architecture for optimizing the number of goroutines using feedback control in this GopherCon 2019 talk. Dive into the importance of dynamic concurrency design for achieving both speed and stability in Go applications. Learn about performance tuning, Go's runtime scheduler, and the complexities of concurrency. Discover how to implement feedback control mechanisms, including PID controllers and dynamic target controllers, to determine the optimal number of goroutines. Examine the concept of elastic semaphores and the architecture of kaburaya. Analyze experimental results from various tasks to understand the effectiveness of this approach in different scenarios. Gain insights into improving Go application performance without relying on specific platform or load conditions.

Intro
Performance tuning in case of pt
Concurrency and complexity
Concurrency and Go
Runtime scheduler of Go
Concurrency and application
Basic idea
Issues to solve for the realization
Performance metrics
Determine the number of goroutines
Feedback control
PID Controller (2/2)
Dynamic Target Controller
To bound concurrency in Go
Elastic semaphore
Architecture of kaburaya
Evaluation
Task: pt mac-scan (Good)
Task: pt mac-no-scan (Bad)
Task: mem_4096000_10000 (Good)
Task: mem_409600_100000 (Not so good)
Experimental results
Conclusion
Reference