Cailin O'Connor

An important and fascinating book. I was surprised, and pleased, that most of the work was conducted at one of my old alma mater’s ...The ANU Research School of Philosophy where my supervisor was resident....Though both authors appear to be based at the University of California.. It’s long been a source of mystery to me how some of my, apparently intelligent and thoughtful friends could continue to hold the views they do in the face of so much authoritative evidence to the contrary. I show more also find it fascinating that some much less educated acquaintances that I meet while exercising my dog, hold radical views on vaccination and diet which don’t seem to be rational. This book sets out a carefully argued and convincing model which can account for these beliefs. Basically it suggests that our beliefs are heavily influences by our social networks.
A difficulty that I have in this current review is that I cannot show their diagrams to explain their network models which I found quite convincing...albeit with one qualifier. The models rely on credence levels for some particular belief. For example I might believe something with 90% credence ...meaning that I’m very sure about it but not 100%. Then I might be exposed to new evidence from an experiment which changes my credence level. The authors, say they have used Baye’s law to calculate the revised weightings but the calculations are not shown. It's clearly complex to assign a probability weighting to the new information and even more complex to do the re-weightings. Maybe they could have shown one example calculation in an appendix. But, I think, they still make their basic points. Maybe a lot more work needs to be done to turn this into a practical tool.
They use their model to explain how correct views can be modified by; incorrect data being fed in; by propaganda; by selective reporting of experimental results (ignoring all the negative results for example); by peer pressure; by appeal to authority; by damaging the reputation of an authority, by industry support for partisan outcomes; by ill intentioned researchers etc. they also draw attention to the fact that modern social media provides the means by which false views can be spread very widely and very fast.
I must confess that whilst I was very impressed with their logic and their basic model, I was less impressed by their recommendations for dealing with the problem off false views being entrenched in the community. As they show, some such views become impervious to change. Fourteen percent of Donald Trump supporters simply refused to accept that photos of attendance at his inauguration rally showed lower numbers of people than other photos. (When the differences were stark in the photos).
I don’t disagree with their recommendations.....that we should stop thinking that the "marketplace of ideas" can effectively sort fact from fiction......That the metaphor of the marketplace of ideas, as an analogue to the free market in economics, is a fiction, and a dangerous one. That those in positions of power or influence must see their speech for what it is— an exercise of power, capable of doing real harm.
It is irresponsible to advocate for unsupported views, and doing so needs to be thought of as a moral wrong, not just a harmless addition to some kind of ideal "marketplace."
This means, :
• First, that scientific communities must adopt norms of publication that decrease the chances of spurious findings..
• Second, scientists need to consider inherent risks when they publish...It is the responsibility of scientists to take into account, the social consequences of their work.
But it almost requires something like a religious change in the community to accept that advocating for unsupported view is a MORAL wrong. This doesn’t mean that it can’t happen. (Littering today is regarded by the community as something like a moral wrong.....but it certainly wasn’t when I was a teenager).
I was very impressed with the book and I’ve included extracts below, partly to try and capture the main arguments but also to help my failing memory. An easy five stars from me.

Introduction: The Vegetable Lamb of Tartary
Mandeville, the text claimed, had travelled through Asia Minor, northern Africa, and into India and had experienced many things unknown in Western Europe. Among these wonders was an Indian tree bearing gourd like fruit, within which could be found tiny lambs, complete with flesh and blood. These reports of lamb-plants, which came to be known as the Vegetable Lamb of Tartary, captured the medieval imagination....The Vegetable Lamb was reported as fact by leading naturalists and botanical scholars, several of whom claimed to have studied it and to have seen its wool. These claims persisted into the seventeenth century....But it took nearly four centuries after Mandeville’s writings appeared for European botanists and biologists to recognize the Vegetable Lamb for a myth.....Are the mechanisms by which such beliefs were formed and spread, even among........experts still present?
About six weeks before Donald Trump was elected president of the United States, a website calling itself ETF News posted a story with the headline “Pope Francis Shocks World, Endorses Donald Trump for President, Releases Statement.”...The article was shared or liked on Facebook 960,000 times between when it was posted and the election....It was the single most-shared election-related news item on Facebook in the three months leading up to the election. By contrast, the most-shared article from a reputable news source during the same period was a Washington Post opinion piece titled “Trump’s History of Corruption Is Mind-Boggling. So Why Is Clinton Supposedly the Corrupt One?”
Had the Pope endorsed Trump, it would have been major news....But it never happened. The papal endorsement was the biggest “fake news” story of the election cycle...An analysis by Craig Silverman at BuzzFeed News found that the top twenty fake news stories in the three months before the election were shared or liked a total of 8.7 million times on Facebook....ETF was arguably the most popular “news” source on the internet during the months before the election. Its stories included accusations that Clinton had sold weapons directly to the Islamic State, that US federal law disqualified her from holding office, that former FBI director James Comey had received millions of dollars from the Clinton Foundation, and that President Barack Obama had cut the pay of US military personnel.
None of these stories was true. Many were not even original but were lifted directly from other fake news websites. The papal endorsement story, for instance, was originally posted on a site called WTOE 5 News, part of a network of fake news sites.....Today’s Vegetable Lambs have become a major political force not only in the US but also in the UK and in Europe.....And it raises a deep and unsettling question: Can democracy survive in an age of fake news?
This is a book about belief. It is a book about truth and knowledge, science and evidence. But most of all, it is a book about false beliefs. Beliefs also matter to decisions we make as a society—decisions concerning economic policy, public health, and the environment, among other topics. Do we restrict automobile emissions? This depends on our beliefs about how these emissions will affect public health and how restrictions will affect economic growth....
If you believe false things about the world, and you make decisions on the basis of those beliefs, then those decisions are unlikely to yield the outcomes you expect and desire.
We need to understand where beliefs come from in the first place. Some come from our personal experience with the world.....How effective is a person at drawing reliable inferences under various circumstances? False beliefs presumably indicate that a person is not drawing the right sorts of inferences.....Many of our beliefs—perhaps most of them—have a more complex origin: we form them on the basis of what other people tell us. We trust and learn from one another.
Is there mercury in large fish after all? Most of us haven’t the slightest idea how to test mercury levels. We must rely on information from others....How do we know whether to trust what people tell us? If someone “of credence” tells you that indubitably the Vegetable Lamb is real—or that the Pope endorsed Donald Trump—do you believe that person? What if every smart, well-educated person you know believes in the Vegetable Lamb? What if your friends post articles about the Vegetable Lamb on social media and loudly joke about the ignorance of Vegetable Lamb Deniers?
When we open channels for social communication, we immediately face a trade-off. If we want to have as many true beliefs as possible, we should trust everything we hear....And if we want to minimize the number of false beliefs we have, we should not believe anything.
Most of us get our false beliefs from the same places we get our true ones...We need to understand the social character of belief—and recognize that widespread falsehood is a necessary, but harmful, corollary to our most powerful tools for learning truths.
The deliberate propagation of false or misleading information has exploded in the past century, driven both by new technologies for disseminating information—radio, television, the internet—Much of this misinformation takes the form of propaganda......Mass-media propaganda has long been a tool of governments to control their own citizens and to influence the political fortunes of their competitors....Often more dangerous—because we are less attuned to it—is industrial propaganda. This runs the gamut from advertising, which is explicitly intended to influence beliefs, to concerted misinformation campaigns designed to undermine reliable evidence.....A classic example of the latter is the campaign by tobacco companies...which successfully delayed, for a generation or more, regulation and public health initiatives to reduce smoking....The methods pioneered by cigarette makers have been emulated by the energy industry and allied scientists and politicians to create an impression of uncertainty concerning the severity and causes of climate change...All of these sources of deliberately partial, misleading, and inaccurate information—from political propaganda, to politically motivated media, to scientific research shaped by industrial interests—play an important role in the origins and spread of false beliefs.....Many, many people over the past century have watched loved ones—smokers—die premature, painful deaths. This is precisely the sort of direct experience that should bear on belief, unless other factors override it....So how can propagandists override the weight of evidence from both direct experience and careful scientific inquiry to shape our beliefs?
In this book we argue that social factors are essential to understanding the spread of beliefs, including—especially—false beliefs....In part, our argument draws on historical (and recent) examples of false beliefs that have spread through communities of people trying to learn about the world. Most of these examples come from science....As we argue, scientists, just like the rest of us, are strongly influenced by their networks of social connections....The sorts of social considerations that we discuss here are crucial to understanding the persistence and spread of virtually all false beliefs.
Scientists are the closest we have to ideal inquirers. For these reasons, the fact that even communities of scientists can persist in false beliefs is striking—We all have experiences, remember them, and change our beliefs in light of those experiences. Scientists, when doing science at least, merely try to be more systematic about this process.
There is a pervasive idea in Western culture that humans are essentially rational, deftly sorting fact from fiction, and, ultimately, arriving at timeless truths about the world...Models of social learning help us see that this picture of human learning and rationality is dangerously distorted.....Individually rational agents can form groups that are not rational at all....Our ability to successfully evaluate evidence and form true beliefs has as much to do with our social conditions as our individual psychology.....While the John Birch Society and their kin were certainly sincere in their opposition to water fluoridation (and many other government activities), they remained firmly on the fringes during the twentieth century.
Today, however, the situation appears to be different. Evidence-poor arguments about public-health issues such as global climate change, vaccination, and genetically modified foods are not only widely discussed and credited in mainstream political discussions, but in many cases they are actively supported by members of the current US administration, members of Congress, and some leading politicians...Over the past two decades, influential figures in American and British public life have adopted an ever-more-tenuous connection to the truth—and a complete disregard for evidence, expert knowledge, or logical coherence—
One of our key arguments in this book is that we cannot understand changes in our political situation by focusing only on individuals. We also need to understand how our networks of social interaction have changed, and why those changes have affected our ability, as a group, to form reliable beliefs.....It is not necessary for propagandists to produce fraudulent results to influence belief. Instead, by exerting influence on how legitimate, independent scientific results are shared with the public, the would-be propagandist can substantially affect the public’s beliefs about scientific facts.....If journalists make efforts to be “fair” by presenting results from two sides of a scientific debate, they can bias what results the public sees in deeply misleading ways.....While it might seem that the solution is more information, this view is too limited. We have more information than ever before. Arguably, it is the abundance of information, shared in novel social contexts, that underlies the problems we face.
One: What Is Truth?
It was widely believed that the problem of ozone depletion was well in hand. No one, including Molina and Rowland, believed that there was any immediate risk of holes opening in the ozone layer....So Richard Stolarski, a physicist working at NASA’s Goddard Space Flight Centre in Maryland, decided to revisit the satellite data for ozone levels over Antarctica....On careful re-evaluation, he was surprised to find that the satellite HAD detected the ozone hole. But no one had noticed. The reason was that the measured levels were so low that the data-processing software had thrown them out as outliers—“Any time scientists run an experiment they need to separate the signal from the noise. Designing software to do this requires a certain artfulness—and a lot of knowledge about the thing you are trying to measure....One of the main contributors to the error that there would be no ozone hole was the fact that the air above Antarctica is so cold that clouds there are composed of ice particles rather than water vapor. It turned out that these ice particles remove nitric acid from the air, which in turn allows the chlorine released by CFCs to persist longer, increasing ozone depletion....Meanwhile, the continent’s weather patterns have a distinctive character: powerful, frigid winds circle the South Pole, forming what is known as a polar vortex.
The nations of the world had acted definitively and with conviction. And they had done so on the basis of sound and exhaustive science. In the end, our scientific process did the best thing we could ask of it: it saved us all from space radiation.
The idea of truth presents many old, difficult philosophical problems. Can we uncover truths about the natural world? Are there reliable methods for doing so? Can we ever really know anything?....As a scientific consensus emerged during the middle part of the 1970s that CFCs posed a serious risk to ozone levels, and US policy makers began to implement regulatory responses, the chemical industry pushed back. Led by DuPont, the massive American chemical manufacturer, industry representatives argued against doing anything.
The problem was that the industry continued to call for more research, and for delayed action, irrespective of how much evidence came in....As late as March 1988, the CEO of DuPont wrote to the US Senate to declare that there was no need for drastic reductions. By this point, it was hard to imagine what further evidence you could ask for. And yet the industry kept asking for more—for certainty.....This has become known as the “Problem of Induction.” Hume concluded that we cannot know anything about the world with certainty, because all inferences from experience fall prey to the Problem of Induction. The fact is that science can always be wrong....And it is not merely that we cannot be certain. Scientists have often been wrong in the past. The history of science is littered with crumpled-up theories that scientists once believed, on the basis of a great deal of evidence, but which they now reject.....For nearly two thousand years, scientists believed bad air, or “miasma,” emanating from rotting organic matter was the chief cause of disease—A thousand years of precision measurements and careful mathematical arguments had established, beyond a shadow of doubt, that the earth stands still and that the sun, planets, and stars all move around the stationary earth—Philosophers of science, such as Larry Laudan and P. Kyle Stanford, have argued that these past failures of science should make us very cautious in accepting current scientific theories as true....Surely caution about accepting new scientific findings is always in order.
We might, for instance, become very, very confident that CFCs are creating a hole in our ozone layer.....But ultimately, we care about truth (at least scientific truth) inasmuch as true beliefs allow us to act successfully in the world....What we want is enough confidence to avoid getting fried by radiation from space. When it comes to the question of what we should do, we need to set general skepticism aside and act on the basis of the evidence we have.
As Hume himself put it, “A wise man . . . proportions his belief to the evidence.”
There is a formula, known as Bayes’ rule, that allows you to calculate what your degree of belief, or credence, should be after learning of some evidence, taking into account what you believed before you saw the evidence and how likely the evidence was....In 1962, Thomas Kuhn, a physicist-turned-historian, published a book called The Structure of Scientific Revolutions...A scientific revolution is a change of paradigm: a radical discontinuity, not only in background theory, but in scientists’ whole way of seeing the world...Kuhn’s work raised the possibility that to understand science, we had to recognize it as a human enterprise, with a complex history and rich sociological features....Reconsidering science as embedded in a broader cultural and political context that could influence scientific thought led to some troubling realizations.
The whole field of statistics emerged when Karl Pearson and Francis Galton (Charles Darwin’s cousin) attempted to quantify various markers of racial superiority..[This is only partly correct....Active in the field shortly after was Ronald Fisher mainly in the field of Agricultural Science and experimental layout and analysis of variance]. .Science was also implicated in colonialism, which had often been justified by “scientific” arguments about racial superiority....It was in this intellectual context that industry advocates raised the concern that scientists researching the ozone hole were themselves political agents, influenced by their background views about environmentalism, government regulation, and the value of industry....It is true that, like all of us, scientists cannot isolate themselves from their cultural contexts...The sorts of cultural critiques of science emerging from science studies can and have been deeply valuable....But the mere observation that a scientist or group of scientists holds certain cultural or political views does not undermine the evidence and arguments they produce....The insights of Galton and Pearson have been developed into a large and invaluable set of tools for analyzing and interpreting data—Likewise, the hundreds of scientific articles written during the 1970s and 1980s providing careful evidence of the role of CFCs in ozone depletion, are not washed away if some or even all of the authors of those articles happen to think that preserving the environment is an admirable goal in itself....Despite the large body of evidence concerning the causes and harms of acid rain, the Reagan Administration did everything it could to prevent action—up to and including tampering with the scientific record....In 1982, George Keyworth, the White House science advisor, commissioned yet another report on acid rain....Nierenberg submitted a draft list of potential panel members to the White House, Fred Singer’s name was not on it.
His primary affiliation seems to have been with the Heritage Foundation, a conservative think tank.....By March 1983, the panel had produced a complete draft of its report....The draft also included a chapter written by Singer—in which he came to startlingly different conclusions. He argued that every course of action bore unacceptable costs, except doing nothing.....The final report did not accurately reflect the panel’s recommendations—and it was released, with panel members’ names, in a form that those members had not reviewed or approved. This was a significant and controversial breach of protocol....In the end, no legislation on acid rain was seriously considered for five more years, until after Reagan had left the White House....The argument is that there is a kind of political interference in science that is apparent in the case of the Acid Rain Peer Review Panel...By the early 1990s there was a broad perception among many scientists, and also some philosophers, politicians, and journalists, that academics in the humanities were agitating to undermine science. These scientists began to push back.
The ”Higher Superstition” book argued that the sociologists and philosophers who purported to analyze science were generally incompetent to evaluate the work they were responding to....The picture of “truth” and “falsity” that we have sketched in this chapter is one according to which our beliefs play a particular role in guiding action. We seek to hold beliefs that are “true” in the sense of serving as guides for making successful choices in the future; we generally expect such beliefs to conform with and be supported by the available evidence.
The real trouble is that most of us are not in a position to independently evaluate, much less collect and analyze, the full set of data.....For that matter, most individual scientists are not in this position either!.....The reason to believe that there was a hole in the ozone layer was not because scientists said there was such a hole; it was because multiple devices, detected substantially reduced ozone levels. And a carefully tested theory was available according to which CFCs could deplete ozone—All of this remains true even if we accept that science is deeply entwined with culture and politics—The real threat is from those people who would manipulate scientific knowledge for their own interests or obscure it from the policy makers who might act on it.
Two: Polarization and Conformity
Beginning around 2000, an American physician named Jane Hightower began to notice a distinctive cluster of symptoms in her patients: hair loss, nausea, weakness, brain fog. These are all associated with mercury poisoning—[But she copped a huge amount of resistance to her ideas]. Ultimately, the discovery of a new link between mercury poisoning and seafood consumption occurred when a community, or network, of scientists and doctors, all sharing ideas and evidence, adopted a new consensus. With time and ever more evidence, she gradually convinced more and more of her colleagues. Today, government agencies around the world are more savvy about the risks of methylmercury poisoning from fish and have issued guidelines to better control exposure.
The kind of reasoning Watson and Crick did with their building blocks is ubiquitous in the sciences. They built a model as an aid to understanding and inference....Bayesian belief updating gives us a model of how individual beliefs change. But as we have just seen in the case of methylmercury, science often needs to be understood on the level of a community, not an individual.....
The framework we focus on [in this book] was introduced in 1998 by economists Venkatesh Bala and Sanjeev Goyal. It is a mathematical.model in which individuals learn about their world both by observing it and by listening to their neighbours. About a decade after Bala and Goyal introduced their model, the philosopher of science Kevin Zollman, now at Carnegie Mellon University, used it to represent scientists and their networks of interaction. We use the model, and variations based on it, much as Zollman did....The basic setup of Bala and Goyal’s model is that there is a group of simple agents—highly idealized representations of scientists, or knowledge seekers—who are trying to choose between two actions....imagine someone faced with two slot machines, trying to figure out which one pays out more often......Over a series of rounds, each scientist in the model chooses one action or the other. They make their choices on the basis of what they currently believe about the problem....Importantly, each scientist develops beliefs based not only on the outcomes of their own actions, but also on those of their colleagues and friends.
We have been using anthropomorphic language, talking about “scientists” who “decide” to “act” on the basis of their “beliefs.” But in fact we are talking about computer simulations—there are no real decisions here, no physical actions, and no minds that could hold beliefs. Instead, we have an abstract network consisting of a collection of “nodes,” each of which may or may not be connected to other nodes by what is called an “edge.” Each node represents a scientist, and each edge connects two scientists who have access to each other’s results....In general, these models tend to converge to the true consensus—that is, the whole network comes to believe that action B is better. But, as we will see, they sometimes go to the false one.
Decisively showing that bacteria cause ulcers ultimately earned the 2005 Nobel Prize for two Australian medical researchers....In fact, the theory that ulcers were caused by bacteria dates back to 1874,....But the bacterial theory was not the only one available. The other possibility, also accepted by many doctors and scientists, was that stomach acid was the culprit.....But then, in 1954, the bacterial theory suffered a devastating setback. Gastroenterologist E. D. Palmer biopsied the stomachs of more than one thousand patients and found no evidence of bacteria at all....But later Marshall managed to isolate and cultivate the new strain, showing definitively that bacteria could live in the human stomach after all.
Ultimately, Warren and Marshall managed to persuade their colleagues that the bacterial theory was right.
How could this happen? One of the most startling findings from the Bala-Goyal models is just how strongly people’s beliefs can influence one another....This means that a successful new belief can spread in a way that would not have been very likely without the ability to share evidence....The social spread of knowledge is a double-edged sword. It gives us remarkable capabilities, as a species, to develop sophisticated knowledge about the world, but it also opens the door to the spread of false belief.....The “Zollman effect,” after Kevin Zollman, who discovered it suggests that if everybody shares evidence, a chance string of bad data can persuade the entire group to abandon the correct theory.
Some temporary diversity of beliefs is crucial for a scientific community....The Zollman effect can help explain how Palmer’s results finding no bacteria in the stomach had such a dramatic effect—
In 1975, Connecticut health officials took Murray’s [a patient with fever and other symptoms] case to Allen Steere, a rheumatologist......Steere’s extensive investigation into the possible causes of the ailment eventually yielded a diagnosis: a new tick-borne illness later named Lyme disease......A few years later, the strain of bacteria responsible was isolated and named Borrelia burgdorferi.....After treatment with antibiotics, many of them regained lives
What happens after Lyme is treated by antibiotics? This question is at the heart of what has become known as the “Lyme wars.” It is the Lyme wars that put Allen Steere’s safety at risk. [death threats.....this is the USA]. Steere, and most professional doctors’ groups and disease control centres, contend that chronic Lyme is actually a combination of other diseases, plus, perhaps, a mysterious post-Lyme syndrome that might involve a continued immune response to Lyme....Most chronic Lyme patients, they point out, do not test positive for the Lyme spirochete.....They refer to evidence showing that Lyme can survive aggressive antibiotic treatment in dogs, mice, and monkeys44 and can subsequently reinfect ticks and other hosts with live spirochetes.....The medical establishment has its own weapons. Patients are denied insurance coverage for expensive treatments that they claim reduce their symptoms.....And one side is putting people’s lives at risk. The only question is which.
In the case of chronic Lyme disease, we see a situation where a scientific community has polarized over a set of scientific beliefs in much the way that some communities polarize over political beliefs.
Political stances are motivated by social values: moral norms, religious beliefs, and beliefs about social and economic justice. We adopt political positions because we want to promote something we value in our country and our lives. Scientific beliefs, on the other hand, are supposed to be value-free.....In an ideal science, thinkers adopt beliefs that are supported by evidence, regardless of their social consequences.....In fact, this is not how science works. Scientists are people; like anyone else, they care about their communities, their friends, and their country. They have religious and political beliefs.....Everybody involved wants to protect and cure the afflicted. Besides having the same values, the two sides in the chronic Lyme case have access, for the most part, to the same evidence.
Do all scientists trust one another equally? Do they consider all other researchers equally reliable?...There is a different rule that can be used to update your beliefs, called “Jeffrey’s rule,” after Princeton philosopher Dick Jeffrey, who proposed it. Jeffrey’s rule takes into account an agent’s degree of uncertainty about some piece of evidence when determining what the agent’s new credence should be. ....Scientists regularly split into polarized groups holding different beliefs, with each side trusting the evidence of only those who already agree with them.....Eventually you have two groups with opposite beliefs who do not listen to each other at all.....Even worse, this sort of polarization is stable: no amount of evidence from the scientists who have adopted the correct belief will be enough to convince those who adopted the wrong belief.....They receive this evidence just as before. They simply do not believe it.
Those who are skeptical of the better theory are precisely those who do not trust those who test it. Even in cases where scientists listen to each other and do not reach stable, polarized outcomes, mistrust among those with different beliefs can produce transient polarization—
There is a large literature, for instance, looking for explanations of polarization in individual psychology. But researchers in this field tend to assume that when two actors look at the same evidence, if they fail to change their beliefs in the same way, then at least one of them must be irrational.....The models of polarization based on Jeffrey’s rule that we have described strongly suggest that psychological biases are not necessary for polarization to result....When it comes to climate change, for instance, the debate is not primarily about whether something is morally right or wrong, or whether an economic policy is just or not. Rather, the disagreement seems to be about whether carbon emissions from human sources actually contribute to changes in weather patterns. This is not a matter of morality or values: either greenhouse gases are affecting the climate, or they are not.
The take-away is that mistrusting those with different beliefs is toxic.
Of course, the opposite can also happen: sometimes, too much trust can lead you astray,
Ultimately, it is best to judge it on its own merits, rather than on the beliefs of those who present it.
Semmelweis [ with hand washing in hospitals] was right about the connection between autopsies and puerperal fever, and the decisions he made on this basis had meaningful consequences. He saved the lives of thousands of infants and women. But his ideas could have saved many more lives if he had been able to convince others of what he knew. In this case, although he communicated his beliefs to other scientists and provided as much evidence as they could possibly desire, his ideas were still rejected, at great cost.
The message from the world was loud and clear: hand-washing dramatically reduces death by puerperal fever. What went wrong? [As an aside, getting doctors to wash hands between patients is on ongoing problem. I know two people who worked in a large modern Japanese hospital and equivalents in Australia and both had problems getting Drs to wash hands despite evidence that it had an impact on health outcomes in these hospitals. My suggestion is to empower the patients to ask the Drs to wash their hands].
Trump claimed to have had the “largest audience to ever witness an inauguration.” But the evidence did not support this. ...Very quickly the White House’s denial of basic facts became the story. Shown photos of attendance, 15 percent of Trump supporters chose the photo with the clearly smaller crowd. They ignored the stark evidence in front of them and agreed with Spicer.....there is a phenomenon known as “conformity bias.”More than a third of study participants agreed with the others [stooges] in the group. They chose to go against the evidence of their own senses in order to conform with what the others in the group did.
UCLA economists have described a phenomenon known as an “information cascade,” by which a belief can spread through a group despite the presence of strong evidence to the contrary. The next investor sees two investors buying GM shares [when the other investor has inside information that Nissan shares will be better] and, perfectly reasonably, infers that the first two people had private reasons to think GM was preferable. On these grounds, that investor might conclude that his or her own private evidence for Nissan is not as strong as the overall evidence for GM. So that investor decides to buy GM....So you can have a group in which almost every member individually would be inclined to make the right judgment but might end up agreeing collectively on the wrong one....Individuals in the stock-trading case are not trying to fit in with the group. They are making rational decisions on the basis of the evidence available to them....Conformity bias can help explain hand-washing by Drs. His [Semmelweis’s] peers—none of whom were washing their hands—ignored him because they all agreed it was absurd to suppose that gentlemen’s hands could transmit disease.
The variations we have discussed so far have been based on the assumption that what each individual cares about is the truth—As the Asch experiment [matching the length of lines] shows, people care about conformity but also about truth....We find that, on average, the greater their tendencies to conform, the more often a group of scientists will take the worse action.....Pressures from their social realm swamp any pressures from the world.
When we have cliques of scientists, conformity within each group keeps them from ever reaching a common consensus. We can even find networks in which everyone holds the true belief—there is a real consensus in belief—but nonetheless, a large portion of scientists perform the worse action as a result of conformist tendencies...Knowing about conformity can also hurt scientists’ ability to trust each other’s statements....Really, we should be looking at the trade-off between the desire to conform and the benefits of successful actions.
In some situations the world pushes back so hard that it is nearly impossible to ignore..When beliefs are not very important to action, they can come to take the role of a kind of social signal. They tell people what group you belong to—and help you get whatever benefits might accrue from membership in that group.
There are a number of widely held, pseudoscientific beliefs about food and health that tend to have few negative consequences for those who hold them......such as the recent fad of “grounding,” based on claims that literally touching the ground provides health benefits as a result of electron transfer between the body and earth. Again, people are not going to be hurt by putting their feet on the ground....When two cliques settle on two different beliefs, those beliefs come to signal group membership......A man who says he does not believe in evolution tells you something not just about his beliefs but about where he comes from and whom he identifies with......The effects of social engagement on our beliefs and behaviours are myriad and complex. Our social networks are our best sources of new evidence and beliefs. But they also open us up to negative social effects.....We have assumed that all of the scientists in our models share real results, and that they are all motivated by the goal of establishing truth......But there are powerful forces in the world whose interests depend on public opinion and who manipulate the social mechanisms we have just described to further their own agendas.
Three: The Evangelization of Peoples
In December 1952, Reader’s Digest published an article titled “Cancer by the Carton,” which presented the growing evidence of a link between cigarette smoking and lung cancer....Soon more evidence came in. During the summer of 1953, a group of doctors at Sloan Kettering Memorial Hospital completed a study in which they painted mice with cigarette tar. The mice reliably developed malignant carcinomas.....(Time magazine ran the story under the title “Beyond Any Doubt.”)....The bad press had immediate consequences..... A massive sell-off in tobacco stocks could be traced to the recent coverage. The industry saw three consecutive quarters of decline in cigarette purchases, beginning shortly after the Reader’s Digest article.....The tobacco industry panicked. Tobacco executives met with PR agencies
to develop a media strategy that could counter a steady march of hard facts and scientific results...The goal was rather to create the appearance of uncertainty: to find, fund, and promote research that muddied the waters....At the core of the new strategy was the Tobacco Industry Research Committee....The TIRC did support research into the health effects of tobacco, but its activities were highly misleading. Its main goal was to promote scientific research that contradicted the growing consensus that smoking kills......And the Tobacco Strategy worked.
The term “propaganda” originated in the early seventeenth century....The Congregation [Sacra Congregation de Propagana Fide] was charged with spreading Roman Catholicism through missionary work across the world and, closer to home, in heavily Protestant regions of Europe...The Congregation’s activities within Europe were more than religious evangelization: they amounted to political subversion...The current meaning of propaganda as the systematic, often biased, spread of information for political ends.
Many of the methods of modern propaganda were developed by the United States during World War I......CPI veteran Edward Bernays synthesized results from the social sciences and psychology to develop a general theory of mass manipulation of public opinion—
The sugar industry invested heavily in supporting and promoting research on the health risks of fat, to deflect attention from the greater risks of sugar......Bernays himself took a rosy view of the role that propaganda, understood to include commercial and industrial information campaigns, could play in a democratic society. In his eyes it was a tool for beneficial social change:
Perhaps Bernays overstated his case.....If he is right, then the very idea of a democratic society is a chimera: the will of the people is something to be shaped by hidden powers, making representative government meaningless......The Tobacco Strategy was wildly successful in slowing regulation and obscuring the health risks of smoking......The Surgeon General did not issue a statement linking smoking to health risks until 1964—many would-be regulators of the tobacco industry were smokers themselves. Clear conflicts of interest arise, independently of any industry intervention.....Subtle sociological factors can also influence smoking habits......smokers often cluster socially: those with smoking friends were more likely to be smokers and vice versa.
To explore in more detail how propagandists can manipulate public belief, we turn once again to the models we looked at in the last chapter. We do this by adding a new group of agents to the model, whom we call policy makers. Like scientists, policy makers have beliefs, and they use Bayes’ rule to update them in light of the evidence they see....When policy makers are connected to only a small number of scientists, they may approach the true belief more slowly, but they always get there eventually (as long as the scientists do).
Now consider what happens when we add a propagandist to the mix.....This agent aims only to persuade the policy makers that action A is preferable—even though, in fact, action B is.
The propagandist does not hold beliefs of their own. Instead, their goal is to communicate only misleading results to all the policy makers.
There is strong evidence that the tobacco industry itself produced research showing a strong link between smoking and lung cancer that it did not publish. Indeed, as we noted, the industry’s own scientists appear to have been convinced that smoking causes cancer as early as the 1950s—and yet the results of those studies remained hidden for decades,
Suppose that in each study, the propagandist takes action B ten times. Whenever this action is successful four times or fewer, they share the results. Otherwise not.....We find that this strategy can drastically influence policy makers’ beliefs. Often, in fact, as the community of scientists reaches consensus on the correct action, the policy makers approach certainty that the wrong action is better.
Worse, this behaviour is often stable, in the sense that no matter how much evidence the scientific community produces, as long as the propagandist remains active, the policy makers will never be convinced of the truth.....Notice that in this model, the propagandist does not fabricate any data. They are performing real science, at least in the sense that they actually perform the experiments they report, and they do so using the same standards and methods as the scientists. They just publish the results selectively.....Selective publication is common in science even without industrial interference. Experiments that do not yield exciting results often go unpublished, or are relegated to minor journals where they are rarely read.....(This practice is sometimes referred to as “publication bias” or the “file drawer effect,” and it causes its own problems for scientific understanding.).....The more scientists the policy makers are connected to, the greater the chance that they get enough evidence to lead them to the true theory.
Perhaps less obvious is that given some fixed amount of funding, how the propagandist chooses to allocate the funds to individual studies can affect their success.....Surprisingly, the propagandist will be most effective if they run and publicize the most studies with as few data points as possible.....A propagandist can share only these studies and mislead policy makers.....Propagandist can use more subtle tools that are both cheaper and, all things considered, more effective.....Selective sharing involves searching for and promoting research that is conducted by independent scientists, with no direct intervention by the propagandist, that happens to support the propagandist’s interests.....Selective sharing was a crucial component of the Tobacco Strategy.....Their newsletter ran headlines such as “Five Tobacco-Animal Studies Report No Cancers Induced” without mentioning how many studies DID report induced cancers....The propagandist does not do science. They just take advantage of the fact that the data produced by scientists have a statistical distribution, and there will generally be some results suggesting that the wrong action is better.....In a large range of cases, a propagandist using only selective sharing can lead policy makers to converge to the false belief even as the scientific community converges to the true one.
The propagandist does especially well when scientists produce many studies, each with relatively little data. How much data is needed to publish a paper varies dramatically from field to field.....Our models suggest that it is better to give large pots of money to a few groups, which can use the money to run studies with more data, than to give small pots of money to many people who can each gather only a few data points.
(All else being equal, studies with more data are said to have higher statistical power, which is widely recognized as essential to rigorous science.) Despite this, low-powered studies seem to be a continuing problem. The prevalence of such studies is related to the so-called replication crisis facing the behavioural and medical sciences. In a widely reported 2010 study, a group of psychologists were able to reproduce only thirty-six of one hundred published results from their field.....Part of the problem is that papers showing a novel effect are easier to publish than those showing no effect. Thus there are strong personal incentives to adopt standards that sometimes lead to spurious, but surprising, results....The success of selective sharing is striking because, given that it is such a minimal intervention into the scientific process, arguably it is not an intervention at all. In fact, in some ways it is even more effective than biased production...It is much cheaper:....It is also less risky, because propagandists who share selectively do not hide or suppress any results....As more scientists work on a problem, the more spurious results they will produce, even if they generally produce more evidence for the true belief.
Holman and Bruner contend that industry can influence science without biasing scientists themselves by engaging in what they call “industrial selection.”....Suppose that some scientists use methods and hold background beliefs that are more likely to erroneously favour action A over action B....The idea is that one could adopt methodologies in science that are not particularly well-tuned to the world, even if, on balance, most methods are.
Some scientists will be more productive than others.....The propagandist finds the scientist whose methods are most favourable for the theory they wish to promote and gives that scientist enough money to increase his or her productivity. This does two things. It floods the scientific community with results favourable to action A, changing the minds of many other scientists. And it also makes it more likely that new labs use the methods that are more likely to favour action A, which is better for industry interests......They do it simply by increasing the amount of work produced by well-intentioned scientists who happen to be wrong....Once scientists have produced a set of impressive results, they are more likely to get funding from governmental sources such as the National Science Foundation.
This case [heart antiarrhythmic drugs] is a situation in which pharmaceutical companies were able to shape medical research to their own ends—the production and sale of antiarrhythmic drugs—without having to bias researchers. Instead, they simply funded the researchers whose methods worked in their favour.....actually leading to massive loss of life......Industrial selection disrupts the workings of the scientific community itself. This is especially worrying, because when industry succeeds in this sort of propaganda, there is no bastion of correct belief.
Industry can also successfully manipulate beliefs within a scientific community if it manages to “buy” researchers who are willing to produce straightforwardly biased science....An embedded propagandist of this sort can permanently prevent scientists from ever reaching a correct consensus. [There is evidence from big pharma that this can be done by subsidising research, travel to conferences, accommodation, books, sponsoring research centres in Universities....not necessarily by paying cash bribes].
But manipulating the evidence we use is not the only way to manipulate our behaviour. For instance, propagandists can play on our emotions, as advertising often does. Poignancy, nostalgia, joy, guilt, and even patriotism are all tools for manipulation that have nothing to do with evidence. The famous Marlboro Man advertising campaign, for instance....One of Bernays’s principal insights, both in his books and in his own advertising and public relations campaigns, was that trust and authority play crucial roles in shaping consumers’ actions and beliefs. This means that members of society whose positions grant them special authority—scientists, physicians, clergy—can be particularly influential. Bernays argued that one can and should capitalize on this influence.
There was no evidence to support the claim that bacon is in fact beneficial [he got a bunch pf physicians to agree that bacon and eggs was a hearty breakfast]...We do not even know what percentage of the physicians he contacted actually agreed with the assertion. But that was of no concern: what mattered was that the strategy moved rashers.....If the right scientific claims can help sales, the wrong ones can decimate an industry—as we saw earlier when the appearance of the ozone hole soon led to a global ban on CFCs.[they seem to have mixed up their wording here....right scientific claims can also decimate an industry].
The harder it becomes for us to identify reliable sources of evidence, the more likely we are to form beliefs on spurious grounds. For precisely this reason, the authority of science and the reputations both of individual scientists and of science as an enterprise are prime targets for propagandists.
Roger Revelle was one of the most distinguished oceanographers of the twentieth century.
Revelle became director of the Scripps Institute of Oceanography....It was widely believed that the carbon dioxide introduced by human activity would be absorbed by the ocean...It was this claim that Revelle and Suess refuted in their article.....Things would only get worse if emissions rates continued to increase—...This work gave scientists good reasons to doubt their complacency about greenhouse gases. But just as important was Revelle’s activism,
Revelle helped Keeling get funding to collect systematic data concerning atmospheric carbon dioxide levels.....In 1965, Revelle moved to Harvard. There he encountered a young undergraduate named Al Gore....[leading to Gore’s subsequent activism].
After a lecture by Revelle it seems that Fred Singer, whose service on the Acid Rain Review Panel we described in Chapter 1, approached Revelle and asked whether he would be interested in co-authoring an article based on the talk.. And in 1991, an article appeared in the inaugural issue of a journal called Cosmos, listing Singer as first author and Revelle as a coauthor. The article asserted (with original emphasis), “We can sum up our conclusions in a simple message: The scientific base for a greenhouse warming is too uncertain to justify drastic action at this time.”....If this sounds identical to Singer’s message on acid rain, that is because it was.)....(Revelle never had a chance to set the record straight: he died on July 15, 1991, shortly after the article appeared in print.)....Singer claimed that Revelle had been a full coauthor, contributing ideas to the final manuscript and endorsing the message. But others disagreed. Both Revelle’s personal secretary and his long-term research assistant claimed that Revelle had been reluctant to be involved and that he contributed almost nothing to the text.
Ultimately, though, what Revelle believed did not matter. The fact that his name appeared on the article was enough to undermine Gore’s environmental agenda....What happened to Gore was a weaponization of reputation....More, although the conclusion of the Cosmos article was regularly quoted, no evidence to support that conclusion was discussed by Easterbrook or Will [journalists] in their articles.....If Revelle had devastating new evidence that led him to change his mind about global warming, surely that should have been presented. But it was not.....The details of how Singer and others used Revelle’s reputation to amplify their message may seem like a special case.....But it shows that how we change our beliefs in light of evidence depends on the reputation of the evidence’s source.
In 2009 Fred Singer, in collaboration with the Heartland Institute, a conservative think tank, established a group called the Nongovernmental International Panel on Climate Change (NIPCC)......The NIPCC produces reports modelled exactly on the IPCC’s reports: the same size and length, the same formatting—and, of course, reaching precisely the opposite conclusions.....The NIPCC looks superficially the same, but of course has nothing like the IPCC’s stature. [Seems to me that Fred Singer seems to be a serial perpetrator of activity contrary to human interests:-anti-fluoridation, pro-smoking, anti-climate change...so is he the equivalent of a war criminal? Presumably he actually thinks he’s doing good by questioning/undermining scientific consensus].
[Another technique is quoting scientists who truly had made major contributions to their respective fields, and their reputations rightly put them in positions to exert influence even in areas where they had far less expertise.. The weaponization of reputation is deeply connected to the polarization models we discussed in the last chapter....But trust based on agreement about the topic at hand does not capture some of the other ways reputation can be weaponized.....We can use our models to capture some interesting and important aspects of the weaponization of reputation by looking at the relationship between trust, belief, and scientific success.
In most real-world cases, we have more to go on than just the problem at hand. [Our] models suggest that one way to influence the opinions of members of a group is to find someone who already agrees with them on other topics and have that person share evidence that supports your preferred position.....We look to people who have been successful in solving other problems and trust them more when evaluating their evidence.
The spread of variolation [for smallpox] had little to do with new knowledge about its success or safety. Instead, it was shunned and then adopted because of social pressures....Later, it was social pressures among nobility to share the beliefs and practices of the princess and her friends that accelerated its spread..Some individuals are connected to disproportionately many people and therefore have outsized social influence. Their actions tend to affect what the many people will do......Todasy, the role of an opinion piece in a major paper is to place someone at the centre of a star, at least briefly.....If a central individual changes belief, that person exerts strong conformist influence on peripheral individuals, who will likely also change their beliefs.....As in Lady Mary’s day, there continue to be people—“ anti-vaxxers”—who question the safety and wisdom of inoculation.....Strikingly, these people tend to cluster in neighbourhoods, both physical and social, in which the discomforts of disagreement over a controversial topic can be avoided.
In 2017, a tight-knit Somali-American community in Minnesota experienced the state’s worst measles outbreak since the 1980s. After learning that rates of severe autism were particularly high in this group, anti-vaccine advocates posted fliers and ads throughout the Somali community centre cautioning against vaccination. They also distributed pamphlets at community health meetings. Andrew Wakefield, the scientist who infamously, and falsely, first reported a link between vaccines and autism, visited Minneapolis repeatedly in 2010 and 2011 to talk with Somali parents of autistic children.....And once many members of the group decide not to vaccinate, the social effects within the group make it much more stable than one bold person bucking a trend.
Four: The Social Network
Emotion plays no role in our models. Neither does intelligence nor political ideology. We have only very simple, highly idealized agents trying to learn about their worlds using (mostly) rational methods. And they often fail. Moreover, they can be readily manipulated to fail, simply by an agent taking advantage of the same social mechanisms that, in other contexts, help them to succeed.
What if these sorts of social factors lie behind the spread of “fake news” and even the bleeding of conspiracy theories into mainstream sources such as the Washington Post and Fox News?.....“Fake news” has a long history, particularly in the United States.....Immediately before and after the American Revolution, partisans on all sides attacked their opponents through vicious pamphlets often filled with lies....Likewise, fake news arguably launched the Spanish American War. [A US warship exploded in Havana harbour....US newspapers blamed Spain....but never any evidence that Spain was involved].
In February 2016, Facebook reported that the 1.59 billion people active on its website are, on average, connected to one another by 3.59 degrees of separation....So even if fake news is not new, it can now spread as never before. This makes it far more dangerous.
Some people do believe fake news. Clearly Edgar Welch, for instance, believed that the Comet Ping Pong pizzeria [where he murdered several people] harboured trafficked children. And he is not alone.
For many issues, focusing on novel or unexpected events is unproblematic. Novelty makes things salient, and salience sells papers....But for some subjects, including science as well as politics and economics, a novelty bias can be deeply problematic....Focusing on only part of the available evidence is a good way to reach the wrong belief....All it takes is a mechanism by which the evidence is selectively disseminated.....This is precisely what happens when journalists focus on novel, surprising, or contrarian stories—She only shares what is most surprising....We find that the public sometimes converges to the false belief, even when the journalist and the scientific community converge to the true one.....Journalism has legal and ethical frameworks that seek to promote “fairness” by representing all sides of a debate...Therefore pressures remain for journalists to present both sides of disagreements (or at least appear to).....Sharing equal proportions of results going in both directions puts a strong finger on the scale in the wrong direction......Ultimately, the mere existence of contrarians is not a good reason to share their views or give them a platform.
Foreign-policy experts, along with politicians across the political spectrum, almost uniformly adopted the view that Saddam Hussein was developing weapons of mass destruction. (He was not.)....The New York Times was widely criticized for presenting this consensus without adequate scrutiny or skepticism, and the editors took the highly unusual step of issuing an apology in 2004.....So we need journalists to avoid sensationalizing new findings and to report both that there is a consensus (when there is one) and the reasons for it. ...This is where institutions can play an important role. Journalists reporting on science need to rely not on individual scientists but on the consensus views of established, independent organizations,
[A DNC researcher was shot on the street] allegations [of hacked DNC emails being found on his computer and leaked to Russians] were attributed to a man named Rod Wheeler, a former Washington, D.C., homicide detective who had been paid to work on the Rich case by a Republican insider. Wheeler talked as if he had himself seen the messages on Rich's computer and could speak directly to this new evidence......There was only one problem: the Fox News story was completely fabricated. Shortly after it appeared, the FBI stated that it was not involved in the Rich investigation. Soon Wheeler admitted that he had not, after all, seen the emails on Rich's computer.....Hannity [Fox News], in particular, has refused to issue a retraction over the Wheeler remarks-even after Wheeler himself disavowed them.
The Rich example shows how thin the line between "real news" and "fake news" can be. Of course, Fox News (like ETF News— and MSNBC) has a transparent political orientation; but by running a story based on the remarks attributed to Wheeler, it veered from editorial slant into blatant falsehood.....A deeper issue concerns a more subtle way in which fake news shades into real news: it sets a journalistic agenda.
We suggest that media which serve as primary news sources, such as the Wall Street Journal, New York Times, and Washington Post, should consider avoiding the first sort of activity [policing facts] altogether. Checking facts and policing the media, while extremely important, are best left to independent watchdogs.
Facebook, Twitter, and Google have revealed that accounts linked to the Russian government spent well over one hundred thousand dollars to purchase political ads, most of which seem to have been designed to create controversy and sow civil discord. Facebook has subsequently revealed that Russian-produced political content reached as many as 126 million US users.....Ultimately, the release of the DNC emails has led to lasting divisions and sustained controversies within the Democratic Party, which in turn have affected Democrats' ability to effectively govern. In other words, the emails have produced discord and mistrust—
Social influence was used to push people to more extreme versions of the views they already held......The picture that emerges is one in which Russian propagandists were highly sensitive to the dynamics of social influence,
It you visit one of the Russian-linked community pages-say, the LGBT United page, the community page mimics a star network, with the page creator at the centre. Rather than try to influence people already at the centre of st show less

The Misinformation Age is a very readable philosophical, historical, and mathematical analysis of the spread of misinformation and, of course, its mirror image, information. The authors employed the Bala-Goyal model, a Bayesian based model which examines the way individuals learn about something by both observing and listening to their compatriots, to examine scientists and their networks of interaction.

This may sound like an excessively dry read – it isn’t. The mathematics is reduced show more to text and diagrams of communication networks. The diagrams illustrate the ways the network changes as the individual’s knowledge (true and false), and interactions with other members of the network changes. They also show what happens when the network is impacted by forces such as propagandists, changes in individual authority, etc.

The authors use examples from history, both general and scientific, to illustrate these various scenarios of network change in practice. The examples are far ranging and include such things as the Hearst newspaper push for war with Spain over the sinking of the battleship Maine, the ostracizing of Semmelweis for suggesting “gentlemen have dirty hands”, the industrial obfuscation concerning CFC’s impact on the ozone layer, the Tobacco Institute’s campaign against the connection between smoking and lung cancer, and much more.

The book text is 186 pages. There are 26 pages of chapter notes, a 35-page bibliography, and an extensive index. I would recommend this book to anyone seeking a better understanding of the whys and wherefores of all forms of communications that are part of today’s world and the roles they play in everyday decisions at all levels of society. show less

More academic than I would have liked.

While reading this book, I watched the Athena Annual Lecture: Why evidence matters from Imperial College London delivered by Join Dame Anne Glover, President of the Royal Scottish Geographical Society. We seem to live in an age where the message to employ critical thinking and the scientific method (think falsifiable and repeatable results) needs to be spread. The message here feels crafted for those drawn into political conspiracies but avoids most often confronting deeply held subjective show more beliefs. Instead, the approach is to discuss and review and analyze scientific or real-world objective fallacies from the Vegetable Lamb of Tartary to the back story on acid rain and the resistance to accepting that stomach ulcers caused by bacteria. The thrust is to illustrate how even scientists come to believe one thing by following their community and non-factual motivators. show less