GPT-SW3: The First Large Generative Language Model for Nordic Languages

Explore the development of GPT-SW3, the pioneering large generative language model for Nordic languages, in this insightful conference talk. Delve into the motivations behind creating the model, examine the challenges and opportunities in data collection and computational resources, and discover practical applications. Learn about the future prospects for developing and implementing large language models for less widely spoken languages. Gain valuable insights from Magnus Sahlgren, PhD and Head of Research for Natural Language Understanding at AI Sweden, as he shares his expertise in computational linguistics, philosophy, and artificial intelligence. The talk covers key topics including the history of language models, general capacity models, the Nordic Pile, data processing, training data breakdown, model size breakdown, and validation projects.

Introduction
What are large language models
The history of language models
General capacity models
The Nordic Pile
Processing the Data
Training Data Breakdown
Model Size Breakdown
Brazilius
Megatron
Restricted Prerelease
Validation Project
Questions