LogFAQs > #934392088

red sox 777 posted...

Another thought experiment for LOTM:

Suppose I predict 100 coin flips. I predict half the coin flips as having a 51% chance of coming out heads and half the flips as having a 51% chance of coming out tails. 100 coins are flipped and the result I said was more likely is right 100 times out of 100.

Do these results suggest my model is worse than if it only predicted 51 coins out of 100 correctly? I don't think so. I think my model was wrong, but the error was that it was actually more powerfully predictive than I gave it credit for.

Assuming I'm understanding your setup correctly and exactly half are the slightly heads biased coins and exactly half are the slightly tails biased coins, we can distinguish those two models by using higher moments if we have enough data. In particular, you've slightly reduced the variance of the outcomes (this is obvious if you push the deviation all the way up so half of the coins are guaranteed to be heads and half are guaranteed to be tails; in that case we're guaranteed to get exactly 50 of each). In principle you could continue to look at higher and higher moments to recreate the entire probability distribution.

---

Congrats to Advokaiser for winning the CBX Guru Challenge!