Nathan Lambert

[Administrator]

Personal website - natolambert.com

youtube.com/@natolambert

x.com/natolambert

linkedin.com/in/natolambert

Projects:

Blog - interconnects.ai

SAIL - Substack Artificial Intelligence Library

The RLHF Book, Reinforcement Learning from Human Feedback

 

[Administrator]

Everything You Wanted to Know About LLM Post-Training, with Nathan Lambert of Allen Institute for AI

Nov 21, 2024

In this episode of The Cognitive Revolution, we dive deep into frontier post-training techniques for large language models with Nathan Lambert from the Allen Institute for AI. Nathan discusses the groundbreaking Tulu 3 release, which matches Meta's post-training performance using the LlAMA base model. We explore supervised fine-tuning, preference-based reinforcement learning, and the innovative reinforcement learning from verifiable reward technique. Nathan provides unprecedented insights into the practical aspects of model development, compute requirements, and data generation strategies. This technically rich conversation illuminates previously opaque aspects of LLM development, achieved by a small team of 10-15 people. Join us for one of our most detailed and valuable discussions on state-of-the-art AI model development.

 

[Administrator]

State of AI in 2026: LLMs, Coding, Scaling Laws, China, Agents, GPUs, AGI | Lex Fridman Podcast

Feb 1, 2026

Nathan Lambert and Sebastian Raschka are machine learning researchers, engineers, and educators. Nathan is the post-training lead at the Allen Institute for AI (Ai2) and the author of The RLHF Book. Sebastian Raschka is the author of Build a Large Language Model (From Scratch) and Build a Reasoning Model (From Scratch).

OUTLINE:
0:00 - Introduction
1:57 - China vs US: Who wins the AI race?
10:38 - ChatGPT vs Claude vs Gemini vs Grok: Who is winning?
21:38 - Best AI for coding
28:29 - Open Source vs Closed Source LLMs
40:08 - Transformers: Evolution of LLMs since 2019
48:05 - AI Scaling Laws: Are they dead or still holding?
1:04:12 - How AI is trained: Pre-training, Mid-training, and Post-training
1:37:18 - Post-training explained: Exciting new research directions in LLMs
1:58:11 - Advice for beginners on how to get into AI development & research
2:21:03 - Work culture in AI (72+ hour weeks)
2:24:49 - Silicon Valley bubble
2:28:46 - Text diffusion models and other new research directions
2:34:28 - Tool use
2:38:44 - Continual learning
2:44:06 - Long context
2:50:21 - Robotics
2:59:31 - Timeline to AGI
3:06:47 - Will AI replace programmers?
3:25:18 - Is the dream of AGI dying?
3:32:07 - How AI will make money?
3:36:29 - Big acquisitions in 2026
3:41:01 - Future of OpenAI, Anthropic, Google DeepMind, xAI, Meta
3:53:35 - Manhattan Project for AI
4:00:10 - Future of NVIDIA, GPUs, and AI compute clusters
4:08:15 - Future of human civilization