HuggingFace and Ray AIR Integration: A Python Developer's Guide to Scaling Transformers

Explore the powerful integration between Hugging Face Transformers and Ray AI Runtime (AIR) in this informative 24-minute talk from Data Council. Discover how to scale model training and data loading beyond a single machine, meeting the computational requirements of advanced Machine Learning models. Dive deep into the implementation and API, learning how to create an end-to-end Hugging Face workflow using Ray AIR, covering everything from data ingestion to fine-tuning, hyperparameter optimization, inference, and serving. Gain insights from industry experts Jules S. Damji, a lead developer advocate at Anyscale Inc and MLflow contributor, and Antoni Baum, a software engineer at Anyscale working on various ML libraries. Enhance your understanding of scaling transformers and learn practical techniques to improve your Python development skills in the realm of distributed Machine Learning systems.

HuggingFace + Ray AIR Integration: A Python Developer’s Guide to Scaling Transformers | AnyScale