What are cost functions?

A cost function is like a scoreboard that tells you how well you're doing at solving a problem.

Imagine you're trying to throw a ball into a basket. Every time you throw the ball, the scoreboard checks where it lands, if it goes in the basket, you get a low score; if it misses completely, you get a high score. The cost function is that scoreboard: it helps you know how close you are to getting the right answer.

Think of the cost function like your teacher checking your math homework. If you got all the answers right, she gives you a 0, that’s the lowest possible cost. But if you made mistakes, she might give you a 5 or even a 10, that's a higher cost.

Just like you use the scoreboard to get better at throwing the ball, we use the cost function to help computers learn. Every time they make a mistake, the cost goes up. Then, they try again, and again, until the cost gets as low as possible, just like you learning to throw the ball into the basket!