State of the Industry - Short-Circuiting Reward Functions

Explore the complex landscape of AI safety and alignment in this 26-minute video lecture. Delve into the risks associated with learned optimization in advanced machine learning systems, examining the Inner Alignment Problem and the potential dangers of self-optimizing machines. Analyze the limitations of AI as presented in "The AI Alignment Problem" and consider the implications of world uncertainty on AI development. Investigate the importance of learning in AI systems, drawing parallels with the dangers of hyperpalatable foods in human behavior. Discuss the challenges of disentangling proximal and distal goals, and contemplate the potential objectives of a superintelligent AI. Evaluate the risks and implications of superintelligence, including its potential to intervene in goal information. Critically assess the flaws in current reasoning surrounding superintelligence containment, providing a comprehensive overview of key concerns in the field of AI safety.

- The risks of learned optimization in advanced machine learning systems
- The Inner Alignment Problem
- The dangers of self-optimizing machines
- The limitations of AI according to "The AI Alignment Problem"
- The Uncertainty of the World
- The importance of learning
- The dangers of hyperpalatable foods
- The cost of disentangling proximal from distal goals
- The goals of a superintelligence
- The possibility of an agent with a rich enough action space
- The dangers of a superintelligence
- The risks of a superintelligence intervening in goal information
- Flaws in the reasoning of superintelligence containment