How do large language models like ChatGPT actually learn?

Large language models like ChatGPT learn by practicing with lots of examples, just like you learn to read by reading many books.

Imagine you have a friend who wants to learn how to cook. Every time they watch a cooking show or try a new recipe, they get better at guessing what ingredients are needed for a dish. That’s kind of like how ChatGPT learns, it practices with millions of sentences and paragraphs from all over the internet.

Think of it like learning to read: when you see the word “cat,” you know it means a furry animal that meows. ChatGPT sees lots of words in many different sentences, and it starts to understand what each word usually means, and how words fit together to make full thoughts.

Every time ChatGPT reads a sentence or answers a question, it gets feedback, like when you get a sticker for doing well on a test. Over billions of these practice sessions, it becomes really good at predicting what word comes next, and that helps it write full sentences and even answer your questions!