How Does The Problem of Multiple Comparisons | NEJM Evidence Work?

Imagine you're playing a game where you guess which hand has a coin, and you get to try 10 times. Sometimes, just by luck, you might guess right even when there's no real pattern. That’s like what happens in multiple comparisons.

When scientists do experiments, they often check many things at once. Let’s say they’re testing 20 different medicines to see which one works best. Each time they test a medicine, it’s like guessing which hand has the coin, and sometimes, just by chance, a medicine might look good even if it doesn’t really work.

Now imagine you're playing with 10 friends, each trying to guess which hand has the coin. Even if only one person actually gets lucky, it might seem like everyone is winning, but most of them just got lucky by chance.

Scientists use a special tool called statistics to tell the difference between real results and ones that happened just by luck. That’s how they avoid being tricked by too many guesses, it's like keeping track of who really wins, not just who guessed right once. Imagine you're playing a game where you guess which hand has a coin, and you get to try 10 times. Sometimes, just by luck, you might guess right even when there's no real pattern. That’s like what happens in multiple comparisons.