What is model-free?

Model-free means learning by doing, without knowing all the rules ahead of time.

Imagine you're playing a game where you don’t know how to win, but you try different moves and see what happens. You don’t need someone to tell you the rules; you just learn from your own experience.

Think about learning to ride a bike. You don’t know all the physics of balance or motion, you just get on, pedal, and feel when you wobble or fall. Every time you fall, you try something new: maybe you move your hands a little, or you pedal faster. Over time, you figure out what works without anyone explaining it to you.

A model-free learner is like that kid on the bike, they don’t need a map of the whole path ahead. They just try things and see what happens. A model-based learner, by contrast, would be someone who studies the road first: they look at the map, plan their route, and maybe even memorize every turn before they start riding.

So, model-free is about learning through experience, not by knowing everything in advance.