Capturing Intelligence at the Level of Thought - Deep Learning and Language Models

Explore the cutting-edge developments in artificial intelligence and deep learning in this 1 hour 31 minute lecture from the Santa Fe Institute. Delve into topics such as large language models, query-based attention, and multiheaded tension. Examine scaling laws, the alignment problem, and mathematical challenges in AI. Gain insights into the limitations of current models and discuss the potential for integrated understanding systems. Analyze the concept of "tips of icebergs" in AI development and consider the missing principles in current approaches. Benefit from the speaker's personal experiences and participate in a thought-provoking discussion on the future of capturing intelligence at the level of thought.

Introduction
Deep Learning
Large Language Models
Query Based Attention
Multiheaded Tension
Model Dimensions
Scaling Laws
Alignment Problem
Math Problem
Diagonal Adjacency
Limitations
The Missing Principle
My Experience
Discussion
Tips of Icebergs
Integrated Understanding System