So much for the predictive wizardry of Nate Silver and Five Thirty-Eight's "83% chance" of a Clinton victory

TalkPro and Con

Join LibraryThing to post.

So much for the predictive wizardry of Nate Silver and Five Thirty-Eight's "83% chance" of a Clinton victory

This topic is currently marked as "dormant"—the last message is more than 90 days old. You can revive it by posting a reply.

1proximity1
Nov 11, 2016, 4:46 am

So much for the predictive wizardry of Nate Silver and Five Thirty-Eight's "83% chance" of a Clinton victory

2PaulFoley
Edited: Nov 11, 2016, 5:27 am

Why? If I'm about to throw a (fair) die, I'd say there's a 83% chance it won't come up a 3; there's nothing wrong with that statement when I do get a 3, is there? (Predicting an election is hardly in the same league, but still ...)

3timspalding
Edited: Nov 11, 2016, 7:14 am

No, you don't get it. Silver's was among the highest Trump chances. He was often accused (example) of inflating Trump's chances to stir up false drama and get ratings. But his math was sound, and he always stressed that chances were just that, and that a Trump victory was well inside a normal polling error.

As he put it on the Friday before the vote "Trump is about 3 points behind Clinton ― and 3-point polling errors happen pretty often." Indeed, as he said before the election, his methods would be proved the most right if Trump in fact won, since he rated the chance far higher than others.

Post-election, we need to figure out why the polls were off. Silver and others are trying to figure that out. But Silver never doubted that polls can be systematically off.

Also, his last pre-election numbers weren't 83%. They were 71%.

4proximity1
Edited: Nov 11, 2016, 11:36 am


Click the map to create your own at 270toWin.com

------------

What I speculated (07 May 2016)

Clinton:

HI (14); WA (12); OR (7); CA (55); AZ (11); NM (5); IL (20); FL (29); VA (13) ; MA (11); RI (4); CT (7); NJ (14) ; DE (3); MD (10); D.C. (3); NY (29); VT (3); NH (4); ME (4)
TOTAL : 248

Trump:

AK (3) ; NV (6); ID (4); MT (3); UT (6); WY (3); CO (9); ND (3); SD (3); NE (5); KS (6); OK (7); TX (38); MN (10); IA (6); MO (10); AR (6); LA (8); WI (10); MI (16); IN (11); OH (18); KY (8); TN (11); MS (6); AL (9); GA (16); PA (20); SC (9); NC (15); WV (5)

TOTAL : 290
-----------
Compare with
New York Times Map 28 July, 2016 :

http://www.nytimes.com/2016/07/28/upshot/donald-trumps-path-what-map-should-demo...

----------------------------------------------------------------------

Politico.com

Election map : final results

width=150 height=150>

Actual Electoral College count :

Clinton 47.7% (228) (20 fewer than my guess)

Trump 47.4% (290) (exactly my guess)

5St._Troy
Nov 11, 2016, 12:06 pm

>4 proximity1: Remind me to check in for your opinion a few years from now.

6cpg
Nov 11, 2016, 1:11 pm

So Silver's genius was demonstrated in 2012 when he called every state correctly. And his genius was demonstrated in 2016 when he only said Trump had a 71% chance of losing. Is there any hypothetical outcome that would cast a shadow on his genius?

7jjwilson61
Nov 11, 2016, 2:09 pm

There were lots of pundits that were saying he was nuts for predicting that Trump had a 30% chance of winning.

8prosfilaes
Nov 11, 2016, 6:32 pm

>6 cpg: I wouldn't say his genius was demonstrated by this election, but if he had a stellar 2012, and had one of the most accurate serious (not "Trump will win, therefore I will stack the numbers until he wins") predictions, I see no need to count him out. Had he predicted a strong win for either side, that would have been a disqualifier, but he correctly predicted this election as too close to clearly call.

9StormRaven
Nov 11, 2016, 7:13 pm

Let's put these numbers on context. Silver predicted that Trump had a ~30% chance of winning. That's about how often a good Major League baseball player gets a hit. Is anyone surprised when a player gets a hit?

10proximity1
Edited: Nov 12, 2016, 1:25 am

These events are always , as British parlance has it, "one-off" complex events. They have zero relationship with rolling dice or with any other easily calculable set of fixed* variables in influencing a result. Elections of this complexity cannot be reduced to probabilistic sets of variables since there is no "history" of identical event-circumstances on which to form a serial review of probabilities.

You either understand this or you don't.

Clinton faced Trump once, for a single and unique vote. This will never "repeat" or afford us any future predictive value.

What Silver does is make what we call "educated guesses." When he's right more than just half the time -- I.e. sheer blind guesses--it's because he's "lucky."

------------------

Of course, below the atomic level, even "fair dice" are an abstraction which have sub-atomic randomness we still cannot calculate or measure. So we treat the die rolls as though they were not influenced by factors which corrupt the random probabilities.

11prosfilaes
Nov 12, 2016, 2:02 am

>10 proximity1: Clinton faced Trump once, for a single and unique vote.

This is false. There were many candidates for US President, including Clinton, Trump, Johnson, Stein, McMullin and Castle. And we won't know until December 19, 2016 who won, because the electoral college is a "one-off" complex event.

12proximity1
Nov 12, 2016, 2:13 am


Would someone else care to explain it to prosfilaes?

I don't teach remedial reading-comprehension.

He does not understand this sentence :

"Clinton faced Trump once, for a single and unique vote."

Good luck.

14proximity1
Edited: Nov 12, 2016, 12:08 pm

Of course, depending on the day one chooses, you can have 538 show either Clinton or Trump as the "likely winner" on November 8th --that's the "poll" which mattered.

it doesn't matter that 538 gave Trump better "chances" of winning on election day when nearly all of their polls had him more or less very likely to lose.

----------

"This isn’t just a case of hindsight bias. It’s tricky to decide what tone to take in an article like this one — after all, we had Hillary Clinton favored. But one of the reasons to build a model — perhaps the most important reason — is to measure uncertainty and to account for risk. If polling were perfect, you wouldn’t need to do this. And we took weeks of abuse from people who thought we overrated Trump’s chances. For most of the presidential campaign, FiveThirtyEight’s forecast gave Trump much better odds than other polling-based models. Our final forecast, issued early Tuesday evening, had Trump with a 29 percent chance of winning the Electoral College.1 By comparison, other models tracked by The New York Times put Trump’s odds at: 15 percent, 8 percent, 2 percent and less than 1 percent. And betting markets put Trump’s chances at just 18 percent at midnight on Tuesday, when Dixville Notch, New Hampshire, cast its votes.

More Politics

"So why did our model — using basically the same data as everyone else — show such a different result? We’ve covered this question before, but it’s interesting to do so in light of the actual election results. We think the outcome — and particularly the fact that Trump won the Electoral College while losing the popular vote — validates important features of our approach. More importantly, it helps to explain why Trump won the presidency.
A small, systematic polling error made a big difference

"Clinton was leading in the vast majority of national polls, and in polls of enough states to get her to 270 electoral votes, although her position in New Hampshire was tenuous in the waning days of the campaign. So there wasn’t any reasonable way to construct a polling-based model that showed Trump ahead.* Even the Trump campaign itself put their candidate’s chances at 30 percent, right about where FiveThirtyEight had him.

"But people mistake having a large volume of polling data for eliminating uncertainty. It doesn’t work that way. Yes, having more polls helps to a degree, by reducing sampling error and by providing for a mix of reasonable methodologies. Therefore, it’s better to be ahead in two polls than ahead in one poll, and in 10 polls than in two polls. Before long, however, you start to encounter diminishing returns."

*: Unless you're USC Dornsife/LA Times--- in which case your polls consistently indicated correctly the candidate which won the office.

15jjwilson61
Nov 12, 2016, 12:09 pm

Sorry, but 538 had Trump at more than 30% for months before the election.

16jjwilson61
Nov 12, 2016, 12:12 pm

The USC poll was just as off in Trump's direction as the other polls were off in Clinton's direction.

17cpg
Edited: Nov 14, 2016, 5:45 pm

I know a little something about probability, but I don't understand the defense of Silver's predictions that is based on them being "only probabilistic". Are there people who predict presidential outcomes based on a claim that the outcome is logically necessary and that this necessity has been logically deduced?

18prosfilaes
Nov 14, 2016, 8:14 pm

>17 cpg: I don't understand the defense of Silver's predictions that is based on them being "only probabilistic".

You're going to have to show the context before I understand what you're talking about here.

Are there people who predict presidential outcomes based on a claim that the outcome is logically necessary and that this necessity has been logically deduced?

Professor Allan Lichtman has 13 keys that apparently logically predict the presidential election.

19cpg
Nov 15, 2016, 9:57 am

>18 prosfilaes:

Throughout his books, Lichtman uses words like "likely" and "probably", not the sorts of words one would use in the logical deduction of a logically necessary result.

20prosfilaes
Nov 15, 2016, 6:14 pm

>19 cpg: I have no idea what you're talking about. As I said in >18 prosfilaes:, "You're going to have to show the context before I understand what you're talking about here."

21cpg
Nov 16, 2016, 9:56 am

>20 prosfilaes:

Silver: "The goal of a probabilistic model is not to provide deterministic predictions". "Jack" on Twitter: "Probabilistic models aren't deterministic." "Cheselle Jan Roldan" at Quora: "His forecasts are always probabilistic, not deterministic." Well, duh. No one who's not just trash-talking claims to be making deterministic predictions, so this doesn't differentiate Silver from anyone else.

But if Silver really wants to emphasize contingency, maybe he should change the name of his website to something like

Tenreplaysof2012wouldprobablyhaveresultedinatleastoneRomneyvictory.com

23proximity1
Nov 16, 2016, 10:10 am


>21 cpg:



Bingo! & Thank you :

>12 proximity1:. I.e. :Would someone else care to explain it to prosfilaes?
I don't teach remedial reading-comprehension.
He does not understand this sentence :
"Clinton faced Trump once, for a single and unique vote."

now "treated."

24prosfilaes
Nov 16, 2016, 6:38 pm

>21 cpg: The example in >18 prosfilaes: didn't give us a probabilistic prediction; he said that Trump would win. Using words like "likely" and "probably" doesn't turn something into a probabilistic prediction.

Silver gave the closest answer of anyone I feel wasn't just trash-talking, besides said professor.

Right now, http://www.nhc.noaa.gov/ is telling us that there's a 30% chance a tropical depression will develop in the Atlantic. Are you going to respond like this if a tropical depression does develop? Probabilities are all about how the world is hard to predict, so we admit that, and give some range of just how likely the experts think something is.

>22 cpg: Mymodelactuallysaidthatmycallingallfivethirtyeightrightinvolvedconsiderableluck.com

I never saw Nathan Silver brag about that. If you looked at the details, given that he gave Florida a 50.1% chance of going Obama at the last moment (which it turns out it did), his prediction was that his calling all the EVs in 2012 election correctly was low probability.

25rrp
Edited: Nov 16, 2016, 7:14 pm

>17 cpg:

I know a little something about probability

Is that a little dangerous something? I think the old saw about quantum theory can be applied to probability. If you can fathom probability without getting dizzy, you don't get it.

Which interpretation of probability do you subscribe to?

http://plato.stanford.edu/entries/probability-interpret/

26rrp
Nov 16, 2016, 7:14 pm

I don't get all this piling on to Nate Silver. If you actually read, or listened, to his stuff, you would know that he was very leery about making any actual "predictions" for this election because it was so far from the norms on which the models he uses are based. The day before the election, he was still saying things like "if Trumps wins" so he seriously considered that outcome a possibility. I personally think 538's commentary and modeling was excellent. His models turned out to be a lot better than any other, as far as I can tell. (Subtle point, a model that correctly predicts one outcome is not necessarily better than one that does not.) Despite the difference in electoral votes, this was a very, very, close run thing and small differences and errors were greatly magnified by the first-past-the-post principle.

27proximity1
Edited: Nov 17, 2016, 2:44 am

>26 rrp:


I don't get all this piling on to Nate Silver. If you actually read, or listened, to his stuff, you would know that he was very leery about making any actual "predictions" for this election because it was so far from the norms on which the models he uses are based. The day before the election, he was still saying things like "if Trumps wins" so he seriously considered that outcome a possibility. I personally think 538's commentary and modeling was excellent.


I don't think my point is that hard to grasp.

Of course Silver made predictions. If he or anyone else wants to argue that "X shall probably be elected" that's one thing.

But to seriously presume to assign a numerical value to your assertion's likelihood is sheer charlatanism. Those numerical values were utterly worthless. The fact that they rose and fell by the day or week--if not the hour--ought to tell us that they are worthless.


His models turned out to be a lot better than any other, as far as I can tell.


If such a pollster can't even predict correctly a binary outcome-- Between X or Y, we predict ___, etc., never mind getting the numerical 'probability' wrong (LOL!)-- then he or she should take up crystal ball gazing or tarot card reading and flim-flam the gullible in the traditional manner because a person could flat out guess and on average get it right half the time, for crying out loud!

The Dornsife USC/L.A. Times polls (alone) consistently had Trump as the winner. So, no, 538 did not
turn out to be a lot better than any other,

28cpg
Edited: Nov 17, 2016, 4:52 pm

>25 rrp: "Which interpretation of probability do you subscribe to?"

I subscribe to the philosophy that everything philosophers touch gets ruined. Probability (in its modern measure-theoretic incarnation) is a beautiful, precise mathematical science that quarantines philosophical questions for those who (in the words of C.S. Lewis) wish to "chew the cud for fifty years over inevitable ignorance".

29rrp
Nov 17, 2016, 7:49 pm

>27 proximity1:, >28 cpg:

I'd like to wager that Hillary Clinton will be the next president of the United States. What odds will you offer me?

30proximity1
Nov 18, 2016, 3:50 am

>29 rrp:

I'm not a bookie.



WikipediA® https://en.wikipedia.org/wiki/Odds#Gambling_odds_versus_probabilities

Gambling odds versus probabilities

Main article: Mathematics of bookmaking

In gambling, the odds on display do not represent the true chances (as imagined by the bookmaker) that the event will or will not occur, but are the amount that the bookmaker will pay out on a winning bet, together with the required stake. For instance, if the bookmaker offers odds of 4:6 against a certain horse winning a race, this means that he'll accept a $6 stake in return for a payoff of $4, plus return of the stake, if the horse wins. If the horse loses, the bookmaker keeps the stake. In formulating the odds to display the bookmaker will have included a profit margin which effectively means that the payout to a successful bettor is less than that represented by the true chance of the event occurring. This profit is known as the 'over-round' on the 'book' (the 'book' refers to the old-fashioned ledger in which wagers were recorded, and is the derivation of the term 'bookmaker') and relates to the sum of the 'odds' in the following way:

In a 3-horse race, for example, the true probabilities of each of the horses winning based on their relative abilities may be 50%, 40% and 10%. The total of these three percentages is 100%, thus representing a fair 'book'. The true odds against winning for each of the three horses are 1-1, 3-2 and 9-1 respectively.

In order to generate a profit on the wagers accepted, the bookmaker may decide to increase the values to 60%, 50% and 20% for the three horses, respectively. This represents the odds against each, which are 4-6, 1-1 and 4-1, in order. These values now total 130%, meaning that the book has an overround of 30 (130−100). This value of 30 represents the amount of profit if the bookmaker accepts bets in the correct proportions on each of the horses. The art of bookmaking is what the bookmaker will take in; for example, $130 in wagers and $100 in return – including stakes – no matter which horse wins.

Making a profit in gambling involves predicting the relationship of the true probabilities to the payout odds. Sports information services are often used by professional and semi-professional sports bettors to help achieve this goal.

The odds or amounts the bookmaker will pay are determined by the total amount that has been bet on all of the possible events. They reflect the balance of wagers on either side of the event, and include the deduction of a bookmaker's brokerage fee ("vig" or vigorish).

Also, depending on how the betting is affected by jurisdiction, taxes may be involved for the bookmaker and/or the winning player. This may be taken into account when offering the odds and/or may reduce the amount won by a player.



31rrp
Edited: Nov 18, 2016, 7:39 am

>31 rrp:

Very Good!

... and I'm not a punter. Not all bets go through bookies though; there are plenty of side bets between interested parties. But let's say, hypothetically, that I'm the bookie, and I offered you odds of 100:1 that Hillary will be the next president. Would you or cpg, hypothetically, place that bet? It's easy money.

>28 cpg:

Don't you find it ironic that statements that philosophy is worthless are themselves philosophical statements? Anyway, I'm interested if you would take those odds too.

32proximity1
Nov 18, 2016, 8:23 am

>31 rrp:

If I understand the hypothesis--

you play the bookie and you offer odds _on_ (Hillary) Clinton's being "the next US president"-- "next" after Barack Obama? After president-elect Trump? or Pence? Which is it?

You'd pay off at $100 for every $1 bet that Clinton won't "be the next president."?

Unless you have some inside scoop on plans by the College of Electors to deliberately throw the vote to Clinton (or the House of representatives) where Hillary is supposed to win, then sure, I'd bet she won't "be the next president."--if I had money to bet.

33rrp
Edited: Nov 18, 2016, 6:26 pm

You are right to seek clarity. After all, if you are not dizzy, you don't understand this. We are not in the airy fairy mathematical world of >28 cpg:. This is the real world of 538, the practical world, where the rubber meets the road, the dice roll and the cards turn.

First to the event. As far as I know, at the time of writing, Barak Obama is the current president so the next president will be the president who comes after Obama. If Hillary is that president we will have outcome \H\. If Hillary is not that president we will have outcome \not H\.

Now to the bet. I am not even hypothetically a bookie. This is a hypothetical contract between you and me. The offer I made to you was as follows. You put x in a pot. I put in 100x. If the event \H\ happens, you own the contents of the pot. If the event \not H\ happens, I own it.

You seem to want to take the \not H\ event side. That's OK, but for this deal you would be the one to put 100x in the pot.

But you changed the deal to switch the outcomes. Now you put x in a pot, I put 100x in the pot. If the event \not H\ happens, you own it. In that case, I would be offering you odds of 100:1 that \not H\. As this is hypothetical, let's go with that for now.

What we have determined is that you are, at least hypothetically, a punter. I would accept the deal if I thought the probability of \H\ is greater than 99% (actually 100/1.01 %). You have accepted this deal and so you must think the probability of \not H\ is greater than 1% (actually 1/1.01 %). You also seem reluctant to find the money to make this deal, which to be honest, looks a pretty good deal for you. I am surprised you don't have the courage of your convictions to take on more risk. Perhaps you are all hat and no cow. Only you know the strength of your own convictions; we can only truly tell by your actions.

Some follow up questions would be: We can tell you think the probability of \not H\ is greater than 1%; what is your interpretation of the word "probability" there? How did you come by that probability? What do you think are the "true chances" that \not H\ will occur and what is mean by "true chances"?

34proximity1
Nov 19, 2016, 2:17 am


"I am surprised you don't have the courage of your convictions to take on more risk. Perhaps you are all hat and no cow. Only you know the strength of your own convictions; we can only truly tell by your actions."

I explained that I'd bet on "not H" -- if I had money to bet. Not everyone does and this includes me. More details than that as to why are none of your business but it has nothing to do with any lack of "the courage of conviction" FFS. it's a bet, not an act of courage.

RE :

" We can tell you think the probability of not H is greater than 1%; what is your interpretation of the word "probability" there?

I'd intend it in this context as a very loosely used term which denotes an intuitive hunch for which no numerical value makes any useful sense.

"How did you come by that probability?"

Sheer intuitive "guesswork."

What do you think are the "true chances" that not H will occur and what is mean by "true chances"?

Clearly they're somewhere between "zero" and "1." But where that is precisely we can only guess.

35rrp
Edited: Nov 20, 2016, 5:53 pm

>34 proximity1:

I am sorry, I though I'd made it clear we were dealing with hypotheticals here, which would include hypothetical currency. We didn't even specify the currency. It could be hypothetical pennies, peanuts or pebbles, it doesn't matter.

What does matter is your internal assessment of the probability. By what you say, I'd have assumed that your assessment of the probability of \not H\ is high. But we judge people, in this case perhaps more than others, by their actions more than their words. There are many ways that people can demonstrate their degree of conviction of some probability, but a nice clean one is to determine what they think the odds would be in a fair bet; to make them put their money (or pebbles if you like) where their mouth is. That you are only prepared to bet (hypothetically) at 100:1 against for \not H\ indicates that you don't have much confidence in your stated conviction.

And we still haven't agreed on what "probability" means.

36rrp
Edited: Nov 20, 2016, 5:51 pm

But to change tack slightly.

Assume there is a deck of 1000 playing cards drawn by an evil daemon from a much larger collection. The task is to predict whether there are more red cards or black cards in the deck without counting them all.

Person A draws four cards from the deck and finds 3 red and 1 black and makes the prediction that there are more red cards in the deck.

Person B draws three sets of four cards from the deck and get the following combinations of (red,black): (2,2), (1,3), (1,3). He also includes person's A draw and so predicts that there are more black cards in the deck.

Does person A or person B have the better method of predicting which color is in the majority?

The 1000 cards in the deck are counted and the final tally is 505 red and 495 black. The deck is consigned to the fire.

Now, does person A or person B have the better method of predicting which color was in the majority?

37lriley
Edited: Nov 20, 2016, 10:13 pm

I would think that the democratic party has other and better things to worry about than Nate Silver's heretofore unblemished track record on predicting presidential races.

38rrp
Nov 21, 2016, 5:57 pm

It's going to take a while to sort out the Democratic party, and the Republican party for that matter, and we here will have little impact on that process.

However, we can do what we can to fight for truth in the post-truth society we live in. Part of the problem is an inability to understand what is and is not a reliable source
of information. Yes, Nate Silver and many others seem to have missed an important shift in the last election. But it was a very close, within a Brexit error.

It is important to understand why the predictions were off and to correct that in future. In general, Silver's methods are evidence based, statistically sound and should not be dismissed. If there is a problem, it's with the individual polls which make up the aggregation and perhaps the weights they apply to those polls.

It's also important to point out the errors of those who seem not to understand the details as we all have a duty to squash all 'post-truths' when we come across them.

39proximity1
Nov 22, 2016, 3:20 am



It is important to understand why the predictions were off and to correct that in future. In general, Silver's methods are evidence based, statistically sound and should not be dismissed. If there is a problem, it's with the individual polls which make up the aggregation and perhaps the weights they apply to those polls.

It's also important to point out the errors of those who seem not to understand the details as we all have a duty to squash all 'post-truths' when we come across them.



LOL!

40rrp
Nov 22, 2016, 9:27 am

Just sad.

41rrp
Nov 23, 2016, 11:25 am

>5 St._Troy:

What do you want? A deal? An alliance? Acceptance of something? State what you're angling for and what you're offering in return.

What do I want? I want my president to be a person of virtue. The cardinal virtues would be wisdom, justice, temperance and courage. A more extensive list would be good sense, good calculation, quick-wittedness, discretion, resourcefulness, piety, honesty, equity, fair dealing, endurance, confidence, high-mindedness, cheerfulness, industriousness, good discipline, seemliness, modesty, and self-control.

In my judgment, Hillary Clinton was the least virtuous of any presidential candidate, apart from Trump who was off the scale.

Do I want a deal? Sure, I offered you one over here. But you declined.

Do I want an alliance? Sure, I would like us to agree on a framework for deciding issues of knowledge and value in the political realm, divorced as far as possible from emotion.

Do I want acceptance of something? Sure, I'd like Trump to reject the use of post-truth politics. By post-truth politics I mean "a political culture in which debate is framed largely by appeals to emotion disconnected from the details of policy, and by the repeated assertion of talking points to which factual rebuttals are ignored. Post-truth differs from traditional contesting and falsifying of truth by rendering it of secondary importance."

What am I offering in return? Wisdom. Virtue.

42rrp
Edited: Nov 23, 2016, 11:25 am

This message has been deleted by its author.

44proximity1
Jan 10, 2017, 9:39 am



>43 cpg:

What do you make of all that comparing?

45cpg
Jan 10, 2017, 10:00 am

>44 proximity1:

Unfortunately, all I made of it this morning was Schadenfreude. Still, Schadenfreude is Freude, right?

46proximity1
Edited: Jan 10, 2017, 10:50 am

{ ;^)

I suspect you're on to the makings of a great country-western song -- auf Deutsch.

If 'tweren't fer Schadenfreude,
I wudn't haben keine freude at all

My baby caught the DB--
Left me a mule to ride

Oh my baby caught the DB,
Left me a mule to ride

I took some Schaden from my Rücksack,
sat by the road and cried...