Keeping Developers Happy with a Fast CI

Explore techniques for optimizing CI performance and reducing developer waiting time in this 28-minute RubyConf 2021 talk. Dive into Shopify's strategies for managing a massive and rapidly growing test suite of over 170,000 Ruby tests. Learn about monitoring, test selection, timeouts, and applying the 80/20 rule to improve CI efficiency. Discover data-driven development approaches, test suite optimization techniques, and methods for maintaining stability while scaling. Gain insights on using mapping to run tests related to PR changes, handling dynamic languages and meta-programming, and leveraging ActiveSupport Notifications. Understand how to compare and evaluate test performance based on factors like failure rate, complexity, churn, and coverage. Get inspired to experiment with load testing and incrementally improve your CI process for happier, more productive developers.

Intro
Test Infrastructure
Growth
Overview Data Driven Development Test Selection Test Optimisation Test Budget
Data driven means progress in an activity is compelled by data, rather than by intuition or by personal experience.
Overview Data Driven Development Test Optimisation
Test Suite Optimisation
Stability
2. Use Map to Run Tests Related to PR Changes
Ruby Files Fixtures Translations Configurations Java Script
Dynamic Language
Meta Programming
ActiveSupport Notifications
Random Failure Rate Complexity Churn Coverage
How to compare?
Rome Wasn't Built In a Day
Experiments
Load Testing
Questions?