Why would anyone let LLMs predict 4 tokens at once? Multi-Token Prediction Explained?

It’s faster to guess four pieces at once instead of one by one, just like when you pick out your favorite snacks from a bag all at once!

When an LLM (like a super smart robot that writes stories or answers questions) predicts 4 tokens at once, it means it's guessing 4 parts of a sentence together, not one at a time. A token is like a word, sometimes it’s a whole word, and sometimes it’s part of a word.

Imagine you're playing a game where you have to guess letters in a word. If you can guess four letters all at once instead of just one, you get closer to the full word much faster! That makes the robot, or the LLM, work quicker and feel more like a real writer.

Think of it like having four friends helping you solve a puzzle together. Each friend guesses part of the answer, and when they all guess at once, the whole picture comes together faster than if each one took turns guessing alone.

So, predicting 4 tokens at once is just a smart way to save time, like grabbing your favorite snacks all in one go! It’s faster to guess four pieces at once instead of one by one, just like when you pick out your favorite snacks from a bag all at once!

When an LLM (like a super smart robot that writes stories or answers questions) predicts 4 tokens at once, it means it's guessing 4 parts of a sentence together, not one at a time. A token is like a word, sometimes it’s a whole word, and sometimes it’s part of a word.