Tractable Control for Autoregressive Language Generation
Offered By: Simons Institute via YouTube
Course Description
Overview
Explore a cutting-edge approach to constrained text generation in this 47-minute lecture by Honghua Zhang from UCLA. Delve into the challenges of generating text that satisfies complex constraints using autoregressive large language models. Learn about the innovative GeLaTo (Generating Language with Tractable Constraints) framework, which utilizes tractable probabilistic models (TPMs) to impose lexical constraints in autoregressive text generation. Discover how distilled hidden Markov models can efficiently compute conditional probabilities to guide GPT2 generation. Examine the state-of-the-art performance achieved by GeLaTo on challenging benchmarks like CommonGen, surpassing strong baselines. Gain insights into the potential of this approach for controlling large language models and the future development of more expressive TPMs. This talk, part of the Probabilistic Circuits and Logic series at the Simons Institute, offers a deep dive into advanced techniques for improving text generation with complex constraints.
Syllabus
Tractable Control for Autoregressive Language Generation
Taught by
Simons Institute
Related Courses
Intro to Deep Learning with PyTorchFacebook via Udacity Natural Language Processing with Sequence Models
DeepLearning.AI via Coursera Deep Learning
Universidad AnĂ¡huac via edX Create a Superhero Name Generator with TensorFlow
Coursera Project Network via Coursera Natural Language Generation in Python
DataCamp