Training Large Language Models: Practices and Research Questions

Explore a comprehensive tutorial on training large language models, covering both pre-training and post-training phases. Delve into best practices at each stage of training, drawing from open models and public research papers. Learn about data curation, training algorithms, and safety mitigations. Gain insights into the pipeline for developing advanced language models and discover open research questions for the next generation of LLMs. Use this tutorial as a starting point to engage in discussions about the future of large language model training.

Training Large Language Models: Practices and Research Questions