Pre-training via Denoising for Molecular Property Prediction

Explore a comprehensive lecture on pre-training techniques for molecular property prediction using denoising methods. Dive into the challenges of limited data in 3D molecular structure analysis and learn about a novel approach that achieves state-of-the-art results. Understand the connection between denoising autoencoders and score-matching, and how this relates to learning molecular force fields from equilibrium structures. Examine experimental results demonstrating improved performance on multiple benchmarks, including the QM9 dataset. Gain practical insights into factors affecting pre-training, such as dataset sizes, model architecture, and the choice of upstream and downstream datasets. Follow along as the speaker covers topics ranging from noisy nodes and oversmoothing to denoising score-matching and model scaling. Conclude with a discussion on future work and participate in a Q&A session to deepen your understanding of this cutting-edge research in molecular machine learning.

- Intro
- TLDR
- The Task: Predicting Molecular Properties
- Pre-training via Denoting
- Background: Noisy Nodes
- Oversmoothing vs. Representation Learning
- Method & Theory: Denoising Score-Matching
- Denoising - Learning a Force Field
- Difference From Score-Based Generative Modelling
- Experiments
- Model Scaling
- Conclusion & Future Work
- Q&A