Automated Acceptance Tests for Terminal Applications

Explore automated acceptance testing for terminal applications in this 40-minute conference talk from linux.conf.au 2020. Learn about the tasty-tmux framework for full automated testing of a terminal email application, including benefits, trade-offs, and solutions to common challenges. Discover the importance of automating timing-sensitive tests, understand the concept of acceptance testing from a user's perspective, and gain insights into implementing automated tests for any terminal platform or application. Delve into topics such as the motivation behind automated testing, pitfalls to avoid, and the advantages of using tmux for testing. Gain valuable knowledge on preventing random failures, improving debugging skills, and increasing confidence in pull requests through comprehensive automated testing practices.

Intro
Terminology Motivation • What is the Test Framework • Pitfalls
ACCEPTANCE TESTING • Test whether the system/feature fullfills it's acceptance criteria • testing from the users point of view (e.g, user stories) can be any type (unit-, integration-, system tests)
HOW IS THE TESTING CARRIED OUT? Manual vs Automated • Humans are very good at finding outliers or differences • automated is "infinitely" repeatable and fast
ACCEPTANCE TESTING TERMINAL APPLICATIONS Selenium doesn't exist for terminal applications • support for command line interface apps, but less for text UIS
UNIT TESTING THE TERMINAL APPLICATION UI usually straight forward for parts
HOW DO WE USE TMUX FOR TESTING
PREVENTATIVE MEASURES
PROTOCOL OCCURRENCE OF RANDOM FAILURES
HAVE CI RUN ALL TESTS BEFORE MERGE
TIME TO DEBUG FAILURES improves debugging skills of your engineers better ability to reason out a complex system
CONFIDENCE THAT THE PULL REQUEST DOES NOT REGRESS THE APPLICATION BEHAVIOUR