LogFAQs > #934398025

LordoftheMorons posted...

Ah okay yeah your setup is different than what I was thinking. Anyway, I'm not sure that your example is in the same category of question. When I specify a model, the outputs are not binary predictions but probability distributions, whereas you seem to be both. Here you're saying that your model gives both a probability and a binary outcome. Is there any relationship between the two (e.g. is the binary outcome determined by saying that the outcome above 50% is the one that it "called")?

Another point (which you kind of brought up earlier) is that 538 is not making 50 independent predictions, though the way they display their data may lead one to believe that. The probability distribution is over sets of 50 outcomes of each state (and those results are correlated). This makes it much more likely that they'll call 49 or 50 states right than we'd naively expect viewing the races as independent.

Yes, technically the binary prediction based on the 51% model is a second model. And since the binary model was created after seeing the outcome of the trials, we must exercise some caution. Although the numbers were designed here to make the evidence overwhelming, retrospective modeling notwithstanding.

And yeah, that's another reason 538's results based on the transformation of its model into binary predictions will usually be more accurate in some sense than the underlying model. But sometimes it will be systematically wrong.

---

September 1, 2003; November 4, 2007; September 2, 2013
Congratulations to DP Oblivion in the Guru Contest!