System Support for Efficient Multi-Resolution Visual Computing on Mobile Systems

Explore system support for efficient multi-resolution visual computing on mobile devices in this tinyML Talks webcast. Delve into challenges and opportunities for mobile operating systems and vision sensing pipeline architectures to flexibly support dynamic multi-resolution workloads. Learn about energy-saving techniques, including sacrificing image resolution when high detail is unnecessary for computer vision and augmented reality tasks. Discover the Banner media framework for seamless resolution reconfiguration and ongoing efforts in multi-resolution visual computing systems. Gain insights into driver-based power optimization, energy-proportionality, and format-oblivious memory management. Understand the importance of rapid reconfigurability at low latencies and expressiveness to meet computational needs with minimal developer burden in mobile systems.

tiny ML. Talks
Vision doesn't always need high-resolution images
We can exploit this if image sensing is energy-proportional
Image sensor power breakdown
Idle power limits energy- proportionality
Driver-based power optimization: (1) Aggressive power management
Driver-based power optimization (2) Pixel clock frequency optimization
Energy-proportionality
However, resolution reconfiguration incurs latency penalty
Hardware is not the culprit
In the operating system, resolution reconfiguration undergoes a sequential procedure inside the media framework which requires the application to invoke several expensive system calls
Aspirations for a reconfigurable media framework
We introduce the Banner media framework
Parallel reconfiguration
Format-oblivious memory management
Banner media framework for seamless resolution reconfiguration
Ongoing efforts in multi-resolution visual computing systems
TinyML for all developers Dataset
Next tiny ML Talks