GPT, ChatGPT, text-generation program, OpenAI Inc., San Francisco, California, USA


S3 E18 John Schulman of OpenAI on ChatGPT: invention, capabilities and limitations

Aug 3, 2023

S3 E18 John Schulman (co-founder OpenAI) on ChatGPT: invention, capabilities and limitations

What's in this episode:
00:00:00 John Schulman
00:01:14 sponsors: Index Ventures and Weights and Biases
00:02:17 what is ChatGPT
00:03:17 how is ChatGPT built
00:05:29 generating text better than any human
00:06:42 tasks which bots don't perform well
00:07:29 maybe this is just a beginning
00:08:36 bringing in Reinforcement Learning (RL)
00:10:48 uses of ChatGPT that are surprising
00:12:32 ChatGPT is good at programming
00:13:30 why hallucinations happen and how to avoid them
00:16:06 training ChatGPT to be aware of hallucinations
00:18:42 retrieval vs. training
00:21:34 single pass training
00:22:50 future in scaling up
00:23:49 closed vs open source
00:27:03 need for another approach to push the development in future
00:30:06 does fine-tuning decrease generalization ability
00:32:42 LLM domination and maybe something more on the horizon
00:37:01 John's curious career path
00:40:47 transition from imitation learning
00:43:10 potential in Academia with a limited budget
00:45:40 interesting problems for Ph.D. students
00:47:54 is there any future in tiny data sets
00:50:24 academia or industry as a Ph.D. student today
00:51:57 being John Schulman
00:55:49 things to do to relax
 
OpenAI DevDay, Opening Keynote

Streamed live on Nov 6, 2023

Join us for the opening keynote from OpenAI DevDay — OpenAI’s first developer conference.

We’re gathering developers from around the world for an in-person day of programming to learn about the latest AI advancements and explore what lies ahead.
 
Back
Top