HomeGroupsTalkZeitgeist
This site uses cookies to deliver our services, improve performance, for analytics, and (if not signed in) for advertising. By using LibraryThing you acknowledge that you have read and understand our Terms of Service and Privacy Policy. Your use of the site and services is subject to these policies and terms.
Hide this

Results from Google Books

Click on a thumbnail to go to Google Books.

The Signal and the Noise: Why So Many…
Loading...

The Signal and the Noise: Why So Many Predictions Fail — but Some… (edition 2012)

by Nate Silver

MembersReviewsPopularityAverage ratingMentions
2,031614,920 (3.89)32
Member:ejp1082
Title:The Signal and the Noise: Why So Many Predictions Fail — but Some Don't
Authors:Nate Silver
Info:Penguin Press HC, The (2012), Edition: 1, Hardcover, 544 pages
Collections:Kindle, Your library, Finished Reading
Rating:****
Tags:None

Work details

The Signal and the Noise: Why So Many Predictions Fail — but Some Don't by Nate Silver

  1. 20
    Thinking, Fast and Slow by Daniel Kahneman (BenTreat)
    BenTreat: Integrates some of the analytical techniques Silver describes with common irrational patterns of decision-making; Kahneman's book explains how to use some of Silver's techniques (and other tools) to avoid making decisions which are not in one's own best interest.… (more)
Loading...

Sign up for LibraryThing to find out whether you'll like this book.

No current Talk conversations about this book.

» See also 32 mentions

English (59)  German (1)  Danish (1)  All languages (61)
Showing 1-5 of 59 (next | show all)
Silver, Nate (2012). The Signal and the Noise: Why So Many Predictions Fail-but Some Don’t. New York: The Penguin Press. 2012. ISBN 9781101595954. Pagine 545. 15,67 €

Un libro molto bello, anche se tutt’altro che perfetto. Un libro pieno di eccessi, più che di difetti: Nate Silver sente il bisogno di scrivere di tutto quello che, nel tempo, lo ha interessato e appassionato – il baseball, il poker, la politica – e di tutti i campi in cui, a suo giudizio, la scienza delle previsioni ha accumulato errori e può fare progressi. Tutto questo fa di The Signal and the Noise un libro monstre, ma al tempo stesso uno dei libri più stimolanti dell’anno.

Nate Silver è salito alla ribalta della cronaca in queste ultime settimane, prima per le polemiche sulle sue previsioni sull’esito delle elezioni americane, e poi per il suo trionfo. Ne ho scritto su questo blog in più occasioni, sia a proposito proprio delle elezioni americane (Nate Silver, il vincitore morale delle elezioni americane), sia discutendo dell’affidabilità delle previsioni meteorologiche private (Le previsioni dei meteorologi privati sono distorte?).

Poiché leggendo il libro mi sono annotato una cinquantina di citazioni, mi asterrei dal fare una vera recensione – i temi del libro e le tesi di Silver emergono con sufficiente chiarezza dalle citazioni stesse. Ma vi regalo un paio di videoclip, dato che Silver è, secondo me, un tipo molto interessante, con una vaga rassomiglianza con Clark Kent, un prototipo del geek e del gay, il che ne ha naturalmente fatto il bersaglio dei sostenitori di Mitt Romney. [In un'intervista a The Observer (Nate Silver: it's the numbers, stupid), a Carole Cadwalladr che gli chiede «What made you more of a misfit, […] being gay or a geek?», risponde: «Probably the numbers stuff since I had that from when I was six.»]

Il primo video è la presentazione del libro:

Il secondo una lunga conversazione (circa un’ora) tenuta a Google pochi giorni fa:

* * *

Prima una curiosità: ho imparato leggendo questo libro (posizione Kindle 261) che la locuzione “information overload” è stata coniata da Alvin Toffler in Future Shock nel 1970.

* * *

Ecco le minacciate citazioni, che operò vi soggerisco di leggere, o almeno di scorrere(riferimento come sempre alle posizioni sul Kindle).

The instinctual shortcut that we take when we have “too much information” is to engage with it selectively, picking out the parts we like and ignoring the remainder, making allies with those who have made the same choices and enemies of the rest. [104]

A prediction was what the soothsayer told you; a forecast was something more like Cassius’s idea.
The term forecast came from English’s Germanic roots, unlike predict, which is from Latin. Forecasting reflected the new Protestant worldliness rather than the otherworldliness of the Holy Roman Empire. [134]

The human brain is quite remarkable; it can store perhaps three terabytes of information. And yet that is only about one one-millionth of the information that IBM says is now produced in the world each day. So we have to be terribly selective about the information we choose to remember. [257]

The printing press changed the way in which we made mistakes. Routine errors of transcription became less common. But when there was a mistake, it would be reproduced many times over, as in the case of the Wicked Bible. [275]

When you can’t state your innocence, proclaim your ignorance. [399]

“The major difference between a thing that might go wrong and a thing that cannot possibly go wrong is that when a thing that cannot possibly go wrong goes wrong it usually turns out to be impossible to get at or repair,” wrote Douglas Adams in The Hitchhiker’s Guide to the Galaxy series. [478]

[…] even if the amount of knowledge in the world is increasing, the gap between what we know and what we think we know may be widening. [828]

“When the facts change, I change my mind,” the economist John Maynard Keynes famously said. “What do you do, sir?” [1169]

Olympic gymnasts peak in their teens; poets in their twenties; chess players in their thirties; applied economists in their forties, and the average age of a Fortune 500 CEO is 55. [1437]

[…] statheads can have their biases too. One of the most pernicious ones is to assume that if something cannot easily be quantified, it does not matter. [1626]

When we can’t fit a square peg into a round hole, we’ll usually blame the peg — when sometimes it’s the rigidity of our thinking that accounts for our failure to accommodate it. Our first instinct is to place information into categories — usually a relatively small number of categories since they’ll be easier to keep track of. (Think of how the Census Bureau classifies people from hundreds of ethnic groups into just six racial categories or how thousands of artists are placed into a taxonomy of a few musical genres.) [1808]

It’s not merely that there is no longer a signal amid the noise, but that the noise is being amplified. [2322]

The statistical reality of accuracy isn’t necessarily the governing paradigm when it comes to commercial weather forecasting. It’s more the perception of accuracy that adds value in the eyes of the consumer. [2326]

Forecasts “add value” by subtracting accuracy. [2335]

With four parameters I can fit an elephant,” the mathematician John von Neumann once said of this problem. “And with five I can make him wiggle his trunk.” [2865]

The government produces data on literally 45,000 economic indicators each year. Private data providers track as many as four million statistics. The temptation that some economists succumb to is to put all this data into a blender and claim that the resulting gruel is haute cuisine. There have been only eleven recessions since the end of World War II. If you have a statistical model that seeks to explain eleven outputs but has to choose from among four million inputs to do so, many of the relationships it identifies are going to be spurious. [3127]

But in fact real management is mostly about managing coalitions, maintaining support for a project so it doesn’t evaporate. [3421]

[…] sophisticatedly simple. [3836]

This is why our predictions may be more prone to failure in the era of Big Data. […] Most of the data is just noise, as most of the universe is filled with empty space. [4258-4266]

We can think of these simplifications as “models,” but heuristics is the preferred term in the study of computer programming and human decision making. It comes from the same Greek root word from which we derive eureka. A heuristic approach to problem solving consists of employing rules of thumb when a deterministic solution to a problem is beyond our practical capacities. [4542]

In many ways, we are our greatest technological constraint. The slow and steady march of human evolution has fallen out of step with technological progress: evolution occurs on millennial time scales, whereas processing power doubles roughly every other year. [4947]

[…] it is not really “artificial” intelligence if a human designed the artifice. [4972. In tema di fibre, si usa distinguere tra fibre artificiali – quelle ottenute modificando fibre naturali, come nel caso della viscosa – e fibre sintetiche – quelle ottenute per sintesi a partire dagli idrocarburi]

As Arthur Conan Doyle once said, “Once you eliminate the impossible, whatever remains, no matter how improbable, must be the truth.” This is sound logic, but we have a lot of trouble distinguishing the impossible from the highly improbable and sometimes get in trouble when we try to make too fine a distinction. [5196: un punto di vista nuovo e stimolante su una citazione che è da anni un mio cavallo di battaglia. La frase è pronunciata da Sherlock Holmes]

In the United States, we live in a very results-oriented society. If someone is rich or famous or beautiful, we tend to think they deserve to be those things. Often, in fact, these factors are self-reinforcing: making money begets more opportunities to make money; being famous provides someone with more ways to leverage their celebrity; standards of beauty may change with the look of a Hollywood starlet. [5519]

Smith’s “invisible hand” might be thought of as a Bayesian process, in which prices are gradually updated in response to changes in supply and demand, eventually reaching some equilibrium. Or, Bayesian reasoning might be thought of as an “invisible hand” wherein we gradually update and improve our beliefs as we debate our ideas, sometimes placing bets on them when we can’t agree. Both are consensus-seeking processes that take advantage of the wisdom of crowds. [5609]

“The market can stay irrational longer than you can stay solvent.” [6066: ancora Keynes]

[…] “the fight between order and disorder,” [6202: è di Didier Sornett]

We could try to legislate our way out of the problem, but that can get tricky. If greater regulation might be called for in some cases, constraints on short-selling — which make it harder to pop bubbles — are almost certainly counter-productive. [6218]

CO2 quickly circulates around the planet: emissions from a diesel truck in Qingdao will eventually affect the climate in Quito. [6285]

Climate refers to the long-term equilibriums that the planet achieves; weather describes short-term deviations from it. [6501]

Uncertainty in forecasts is not necessarily a reason not to act — the Yale economist William Nordhaus has argued instead that it is precisely the uncertainty in climate forecasts that compels action […] [6716]

When we advance more confident claims and they fail to come to fruition, this constitutes much more powerful evidence against our hypothesis. We can’t really blame anyone for losing faith in our forecasts when this occurs; they are making the correct inference under Bayesian logic. [6855]

The fundamental dilemma faced by climatologists is that global warming is a long-term problem that might require a near-term solution. [6864]

In science, progress is possible. In fact, if one believes in Bayes’s theorem, scientific progress is inevitable as predictions are made and as beliefs are tested and refined. […]
In politics, by contrast, we seem to be growing ever further away from consensus. The amount of polarization between the two parties in the United States House, which had narrowed from the New Deal through the 1970s, had grown by 2011 to be the worst that it had been in at least a century. […]
The dysfunctional state of the American political system is the best reason to be pessimistic about our country’s future. Our scientific and technological prowess is the best reason to be optimistic. [6913-6930]

To Wohlstetter, a signal is a piece of evidence that tells us something useful about our enemy’s intentions; this book thinks of a signal as an indication of the underlying truth behind a statistical or predictive problem. Wohlstetter’s definition of noise is subtly different too. Whereas I tend to use noise to mean random patterns that might easily be mistaken for signals, Wohlstetter uses it to mean the sound produced by competing signals. [6999]

[…] signal detection vs. signal analysis [7023]

There is a tendency in our planning to confuse the unfamiliar with the improbable. The contingency we have not considered seriously looks strange; what looks strange is thought improbable; what is improbable need not be considered seriously. [7035: è una citazione di Thomas Schelling]

[T]here are known knowns; there are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns—there are things we do not know we don’t know.—Donald Rumsfeld [7060]

[…] detecting a terror plot is much more difficult than finding a needle in a haystack, he said, and more analogous to finding one particular needle in a pile full of needle parts. [7177]

The more often you are willing to test your ideas, the sooner you can begin to avoid these problems and learn from your mistakes. Staring at the ocean and waiting for a flash of insight is how ideas are generated in the movies. In the real world, they rarely come when you are standing in place. Nor do the “big” ideas necessarily start out that way. It’s more often with small, incremental, and sometimes even accidental steps that we make progress. [7593]

Sanford J. Grossman and Joseph E. Stiglitz, “On the Impossibility of Informationally Efficient Markets,” American Economic Review, 70, 3 (June 1980), pp. 393–408. http://www.math.ku.dk/kurser/2003-1/invfin/GrossmanStiglitz.pdf. [9526]

A conspiracy theory might be thought of as the laziest form of signal analysis. As the Harvard professor H. L. “Skip” Gates says, “Conspiracy theories are an irresistible labor-saving device in the face of complexity.” [11798] ( )
  Boris.Limpopo | Apr 29, 2019 |
This text comes as a gulp of fresh air after reading Taleb. The author attempts to find out in which areas predictions work, in which don’t and why. The wide range of areas is covered: weather, seismology, sport bets, economy and political foresights. He shows both successes and failures and tries to find out what are the root causes of them. ( )
  Oleksandr_Zholud | Jan 9, 2019 |
3.5 stars...different from what I expected but still interesting. Definitely still relevant in 2017, maybe even more so. ( )
  3njennn | Nov 25, 2018 |
Would’ve preferred less math and more philosophy ( )
  aechipkin | Jun 27, 2018 |
Very interesting and easy audiobook to listen to, and also one I think would benefit from repeated listening's (every few years or so).... it's great to remind ourselves on occasion how much statistics and predictions can help us and how we can fool ourselves so easily with them also. This book contains a lot of examples, from baseball, to politics, to weather forecasting, from chess to poker to the economy....why some things are predictable with the right information and some are just really guesses that tell us nothing but what we want to see. Definitely an appropriate read in an election year and a reminder to ground myself in the 'real signal' and distinguish it from, and tune out all 'the noise' that is really just a distraction. The old GIGO (Garbage in, garbage out) for a new data rich age. ( )
  RivetedReaderMelissa | Mar 22, 2018 |
Showing 1-5 of 59 (next | show all)
The first thing to note about The Signal and the Noise is that it is modest – not lacking in confidence or pointlessly self-effacing, but calm and honest about the limits to what the author or anyone else can know about what is going to happen next. Across a wide range of subjects about which people make professional predictions – the housing market, the stock market, elections, baseball, the weather, earthquakes, terrorist attacks – Silver argues for a sharper recognition of "the difference between what we know and what we think we know" and recommends a strategy for closing the gap.
added by eereed | editGuardian, Ruth Scurr (Nov 9, 2012)
 
What Silver is doing here is playing the role of public statistician — bringing simple but powerful empirical methods to bear on a controversial policy question, and making the results accessible to anyone with a high-school level of numeracy. The exercise is not so different in spirit from the way public intellectuals like John Kenneth Galbraith once shaped discussions of economic policy and public figures like Walter Cronkite helped sway opinion on the Vietnam War. Except that their authority was based to varying degrees on their establishment credentials, whereas Silver’s derives from his data savvy in the age of the stats nerd.
added by eereed | editNew York Times, Noam Scheiber (Nov 2, 2012)
 
A friend who was a pioneer in the computer games business used to marvel at how her company handled its projections of costs and revenue. “We performed exhaustive calculations, analyses and revisions,” she would tell me. “And we somehow always ended with numbers that justified our hiring the people and producing the games we had wanted to all along.” Those forecasts rarely proved accurate, but as long as the games were reasonably profitable, she said, you’d keep your job and get to create more unfounded projections for the next endeavor.......
added by marq | editNew York Times, LEONARD MLODINOW (Oct 23, 2012)
 
In the course of this entertaining popularization of a subject that scares many people off, the signal of Silver’s own thesis tends to get a bit lost in the noise of storytelling. The asides and digressions are sometimes delightful, as in a chapter about the author’s brief adventures as a professional poker player, and sometimes annoying, as in some half-baked musings on the politics of climate change. But they distract from Silver’s core point: For all that modern technology has enhanced our computational abilities, there are still an awful lot of ways for predictions to go wrong thanks to bad incentives and bad methods.
added by eereed | editSlate, Matthew Yglesias (Oct 5, 2012)
 
Mr. Silver reminds us that we live in an era of "Big Data," with "2.5 quintillion bytes" generated each day. But he strongly disagrees with the view that the sheer volume of data will make predicting easier. "Numbers don't speak for themselves," he notes. In fact, we imbue numbers with meaning, depending on our approach. We often find patterns that are simply random noise, and many of our predictions fail: "Unless we become aware of the biases we introduce, the returns to additional information may be minimal—or diminishing." The trick is to extract the correct signal from the noisy data. "The signal is the truth," Mr. Silver writes. "The noise is the distraction."
 

» Add other authors (12 possible)

Author nameRoleType of authorWork?Status
Nate Silverprimary authorall editionscalculated
Chamberlain, MikeNarratorsecondary authorsome editionsconfirmed
Dewey, AmandaDesignersecondary authorsome editionsconfirmed
You must log in to edit Common Knowledge data.
For more help see the Common Knowledge help page.
Series (with order)
Canonical title
Original title
Alternative titles
Original publication date
People/Characters
Important places
Important events
Related movies
Awards and honors
Epigraph
Dedication
To Mom and Dad
First words
Introduction

This is a book about information, technology, and scientific progress.
1
A CATASTROPHIC FAILURE
OF PREDICTION


It was October 23, 2008.
Quotations
Last words
(Click to show. Warning: May contain spoilers.)
(Click to show. Warning: May contain spoilers.)
Disambiguation notice
Publisher's editors
Blurbers
Publisher series
Original language
Canonical DDC/MDS
Book description
"Nate Silver's The Signal and the Noise is The Soul of a New Machine for the 21st century." —Rachel Maddow, author of Drift

Nate Silver built an innovative system for predicting baseball performance, predicted the 2008 election within a hair’s breadth, and became a national sensation as a blogger—all by the time he was thirty. He solidified his standing as the nation's foremost political forecaster with his near perfect prediction of the 2012 election. Silver is the founder and editor in chief of FiveThirtyEight.com.

Drawing on his own groundbreaking work, Silver examines the world of prediction, investigating how we can distinguish a true signal from a universe of noisy data. Most predictions fail, often at great cost to society, because most of us have a poor understanding of probability and uncertainty. Both experts and laypeople mistake more confident predictions for more accurate ones. But overconfidence is often the reason for failure. If our appreciation of uncertainty improves, our predictions can get better too. This is the “prediction paradox”: The more humility we have about our ability to make predictions, the more successful we can be in planning for the future.

In keeping with his own aim to seek truth from data, Silver visits the most successful forecasters in a range of areas, from hurricanes to baseball, from the poker table to the stock market, from Capitol Hill to the NBA. He explains and evaluates how these forecasters think and what bonds they share. What lies behind their success? Are they good—or just lucky? What patterns have they unraveled? And are their forecasts really right? He explores unanticipated commonalities and exposes unexpected juxtapositions. And sometimes, it is not so much how good a prediction is in an absolute sense that matters but how good it is relative to the competition. In other cases, prediction is still a very rudimentary—and dangerous—science.

Silver observes that the most accurate forecasters tend to have a superior command of probability, and they tend to be both humble and hardworking. They distinguish the predictable from the unpredictable, and they notice a thousand little details that lead them closer to the truth. Because of their appreciation of probability, they can distinguish the signal from the noise.

With everything from the health of the global economy to our ability to fight terrorism dependent on the quality of our predictions, Nate Silver’s insights are an essential read.
Haiku summary

Amazon.com Amazon.com Review (ISBN 159420411X, Hardcover)

Amazon Best Books of the Month, September 2012: People love statistics. Statistics, however, do not always love them back. The Signal and the Noise, Nate Silver's brilliant and elegant tour of the modern science-slash-art of forecasting, shows what happens when Big Data meets human nature. Baseball, weather forecasting, earthquake prediction, economics, and polling: In all of these areas, Silver finds predictions gone bad thanks to biases, vested interests, and overconfidence. But he also shows where sophisticated forecasters have gotten it right (and occasionally been ignored to boot). In today's metrics-saturated world, Silver's book is a timely and readable reminder that statistics are only as good as the people who wield them. --Darryl Campbell

(retrieved from Amazon Thu, 12 Mar 2015 18:14:18 -0400)

(see all 2 descriptions)

Silver built an innovative system for predicting baseball performance, predicted the 2008 election within a hair's breadth, and became a national sensation as a blogger. Drawing on his own groundbreaking work, Silver examines the world of prediction.

(summary from another edition)

» see all 3 descriptions

Quick Links

Popular covers

Rating

Average: (3.89)
0.5
1 2
1.5 1
2 19
2.5 2
3 85
3.5 28
4 179
4.5 26
5 90

Penguin Australia

2 editions of this book were published by Penguin Australia.

Editions: 0141975652, 1846147735

Is this you?

Become a LibraryThing Author.

 

About | Contact | Privacy/Terms | Help/FAQs | Blog | Store | APIs | TinyCat | Legacy Libraries | Early Reviewers | Common Knowledge | 134,818,505 books! | Top bar: Always visible