Nathan Lambert and Sebastian Raschka are machine learning researchers, engineers, and educators. Nathan is the post-training lead at the Allen Institute for AI (Ai2) and the author of The RLHF Book. Sebastian Raschka is the author of Build a Large Language Model (From Scratch) and Build a Reasoning Model (From Scratch).
OUTLINE:
0:00 - Introduction
1:57 - China vs US: Who wins the AI race?
10:38 - ChatGPT vs Claude vs Gemini vs Grok: Who is winning?
21:38 - Best AI for coding
28:29 - Open Source vs Closed Source LLMs
40:08 - Transformers: Evolution of LLMs since 2019
48:05 - AI Scaling Laws: Are they dead or still holding?
1:04:12 - How AI is trained: Pre-training, Mid-training, and Post-training
1:37:18 - Post-training explained: Exciting new research directions in LLMs
1:58:11 - Advice for beginners on how to get into AI development & research
2:21:03 - Work culture in AI (72+ hour weeks)
2:24:49 - Silicon Valley bubble
2:28:46 - Text diffusion models and other new research directions
2:34:28 - Tool use
2:38:44 - Continual learning
2:44:06 - Long context
2:50:21 - Robotics
2:59:31 - Timeline to AGI
3:06:47 - Will AI replace programmers?
3:25:18 - Is the dream of AGI dying?
3:32:07 - How AI will make money?
3:36:29 - Big acquisitions in 2026
3:41:01 - Future of OpenAI, Anthropic, Google DeepMind, xAI, Meta
3:53:35 - Manhattan Project for AI
4:00:10 - Future of NVIDIA, GPUs, and AI compute clusters
4:08:15 - Future of human civilization
