Hugging Face Workshops - Pretraining Language Models & CodeParrot
Offered By: HuggingFace via YouTube
Course Description
Overview
Explore pretraining language models and Hugging Face's CodeParrot in this live workshop led by Leandro and Merve. Dive into the intricacies of data transformation, batching, and deep speed techniques. Learn about CodeParrot's capabilities for SQL and its evaluation process. Tackle coding challenges, address issues with duplicates, and understand deduplication methods. Gain insights into logging, model training loops, clipping, and checkpoints. Discover the potential of crosslingual transfer in natural language processing.
Syllabus
Intro
CodeParrot Overview
Pretraining Language Models
Data Transformation
CodeParrot
CoParrot
Batching
Iter
Tensor
Deep Speed
BigQuery vs DataSets
CodeParrot for SQL
Evaluation of CodeParrot
Coding Challenges
Problems with Duplicates
Deduplication
Questions
Logging
Models
Training Loop
Clipping
Checkpoints
More Questions
Crosslingual Transfer
Taught by
Hugging Face
Related Courses
Interprofessional Healthcare InformaticsUniversity of Minnesota via Coursera Data Science at Scale - Capstone Project
University of Washington via Coursera Implementing ETL with SQL Server Integration Services
Microsoft via edX Introduzione a R
University of Modena and Reggio Emilia via EduOpen Практики работы с данными средствами Power Query и Power Pivot
Saint Petersburg State University via Coursera