Building a Canary Testing Framework

Explore the concept of canary testing frameworks for distributed systems in this 42-minute conference talk from Strange Loop. Learn why canary tests are crucial for detecting issues and regressions in interdependent services, and how they differ from traditional end-to-end tests and monitoring tools. Discover the architecture and components of a canary framework, including test execution, result storage, and alerting mechanisms. Understand the benefits of using Go for framework development and JavaScript for writing test scripts. Gain insights into implementing standard library functions, saving test results to a database, and visualizing test output. By the end of this talk, acquire the knowledge to build and implement a robust canary testing framework to ensure the reliability of your distributed systems.

Building a Canary Testing Framework
What is a canary?
Canary tests should mimic user behavior and test a specific feature.
Allow for more specificity and granularity for the execution environment
Visualization of Test Output / Execution Status
Recording and alerting on metrics via Prometheus
Canary Framework Architecture Components
A database for storing test execution results
Why Go for the actual framework?
Decent support for embedding languages
Allows us to write and run test scripts using JavaScript
The Canary Daemon
Executes the config file and registers the tests to run
Repeatedly runs each test using the runner, save results to the database
Loading Test Configurations
Standard Library Functions
Saving Test Execution Results to a Database
Questions?