Modeling Hierarchies of Sound for Audio Source Separation and Generation - 2024 JSALT

Explore the intricacies of hierarchical sound modeling in this comprehensive lecture on audio source separation and generation. Delve into the multi-level granularity of sound signals, from complex music compositions to individual frequency components. Discover how incorporating prior knowledge of hierarchical relationships enhances explainability and controllability in data-driven audio signal processing models. Learn about innovative approaches to audio source separation, enabling novel control mechanisms for interacting with these models. Examine the importance of multi-granular features in detecting and quantifying training data memorization in large text-to-audio diffusion models. Gain insights into the use of classifier probes to understand a large music transformer's knowledge of music and develop fine-grained, interpretable controls. This in-depth talk provides a thorough exploration of cutting-edge techniques in audio processing and generation, offering valuable knowledge for researchers and practitioners in the field.

2024 JSALT Gordon Wichern, Modeling Hierarchies of Sound for Audio Source Separation and Generation