Fast Language Generation by Finetuning Pretrained Transformers

Explore a cutting-edge approach to improving language generation efficiency in this 31-minute talk from the Toronto Machine Learning Series. Dive into the research presented by Jungo Kasai, a Ph.D. student from the University of Washington, as he discusses a novel method to enhance the performance of large-scale transformer models. Learn about the swap-then-finetune procedure, which converts pretrained transformers into recurrent neural networks (RNNs) to reduce generation overhead while maintaining accuracy. Discover how this technique provides an improved trade-off between efficiency and accuracy compared to standard transformers and other recurrent variants. Gain insights into the lower training costs associated with this finetuning process and understand its potential impact on natural language processing tasks that rely on large-scale pretrained transformers.

Fast Language Generation by Finetuning Pretrained Transforme