Repetition, Adaptation and Language Modeling - 1999

Explore the concept of repetition in language modeling through this 1-hour 8-minute lecture by Kenneth Ward Church from AT&T Labs-Research. Delve into the introduction of adaptive language models and their role in accounting for word bursts, particularly in content-rich words like proper nouns and technical terminology. Examine the phenomenon of burstiness in words and n-grams, comparing the repetition patterns of content-heavy terms to more common words. Investigate how measures from Information Retrieval, such as term frequency and document frequency, can be used to predict both the average frequency and variance of words. Gain insights into the focus of adaptive language model literature on first moments while highlighting the often-overlooked importance of second moments in language analysis.

Repetition, Adaptation and Language Modeling – Kenneth Ward Church (AT&T Labs-Research) - 1999