RT Troubles, Lessons Learned, and Open Questions in Real-Time Systems

Explore real-time (RT) system challenges and solutions in this 46-minute Linux Foundation conference talk. Delve into trouble areas, symptoms, and diagnostic tools like Kernel Shark and CPU Latency analysis. Examine write buffer issues, data path complexities, and internet driver concerns. Learn about mitigation strategies including forced readback, vacation timeouts, and soft IRQ hacks. Investigate HR timers, locking primitives, and priority inversion problems. Discuss the need for IRQs, threat identification, and queue management. Analyze watchdog examples and potential solutions. Cover security considerations, clock sources, and trace points. Gain insights into reboot testing and unarmed systems, equipping yourself with valuable knowledge for addressing RT system complexities.

Introduction
Agenda
Problem space
First trouble area
Symptoms
Kernel Shark
CPU Latency
Write Buffer
Data Path
Internet Driver
What can we do
Forced readback
Vacation
Timeouts
Soft IRQ hack
Sebastians patch
HR timers
Jealousy
Locking primitives
Conditional variables
Priority inversion
Discussion
Disclaimer
Need for IRQ
Threat ID
Interrupts
Queues
Priority inversions
Watchdog example
Possible solutions
Summary
Unarmed
Orange trace
Clock time
Security
Clock sources
Trace points
Reboot test