Steven Sloman

It all begins with toilets.
Everyone (throughout the developed world!) is familiar with toilets. A typical flush toílet has a ceramic bowl filled with water. When the handle is depressed, or the button pushed, the water—and everything that’s been deposited in it—gets sucked into a pipe and from there into the sewage system. But how does this actually happen?
In a study, graduate students were asked to rate their understanding of everyday devices, including toilets, zippers, and cylinder show more locks. They were then asked to write detailed, step-by-step explanations of how the devices work, and to rate their understanding again. Apparently, the effort revealed to the students their own ignorance, because their self-assessments dropped.
(Toilets, it turns out, are more complicated than they appear!)

Sloman and Fernbach see this effect, which they call the “illusion of explanatory depth,” just about everywhere.

People believe that they know way more than they actually do.

What allows us to persist in this belief is other people. In the case of my toilet, someone else designed it so that I can operate it easily. This is something humans are very good at. We’ve been relying on one another’s expertise ever since we figured out how to hunt together, which was probably a key development in our evolutionary history. So well do we collaborate, Sloman and Fernbach argue, that:
“We can hardly tell where our own understanding ends and others’ begins. One implication of the naturalness with which we divide cognitive labor is that there’s ‘no sharp boundary between one person’s ideas and knowledge’ and ‘those of other members’ of the group.”

So not just rationality but the very idea of individual thinking is a myth. Humans rarely think for themselves. Rather, we think in groups. We think we know a lot, even though individually we know very little, because we treat knowledge in the minds of others as if it were our own.
This is not necessarily bad. Our reliance on groupthink have us an edge over all other animals and turned us into the masters of (this) planet. The knowledge illusion enables us to go through life without being caught in an impossible effort to understand everything ourselves. From an evolutionary perspective, trusting in the knowledge of others has worked extremely well for humans.
This borderlessness, or, if you prefer, confusion, is also crucial to what we consider progress. As people invented new tools for new ways of living, they simultaneously created new realms of ignorance; if everyone had insisted on, say, mastering the principles of metal-working before picking up a knife, the Bronze Age wouldn’t have amounted to much. When it comes to new technologies, incomplete understanding is empowering.
But…the knowledge illusion certainly has its downside. The world is becoming ever more complex, and people fail to realize just how ignorant they are of what’s going on. Consequently, some who know next to nothing about meteorology or biology nevertheless conduct fierce debates about climate change (trump/trumpsters), while others hold extremely strong views about what should be done in Iraq or Ukraine without being able to locate them on a map. Also, It gets much more complicated in the political domain. How could we then vest authority in voters and customers who are so ignorant and susceptible to manipulation? If Sloman and Fernbach are correct, providing future voters and customers with more and better facts would hardly solve the problem. (Try using facts and proofs to convince half-witted, ignorant and imbecile Trump and Trumspters that climate change is actually a thing (and many other similar things that are established facts), and not a propaganda by China)
Encouraging people to be more realistic about their ignorance is, as it sounds, very hard!
People rarely appreciate their ignorance, because they lock themselves inside an echo chamber of like-minded friends, and self-confirming news-feeds, where their beliefs are constantly reinforced and seldom challenged.

“As a rule, strong feelings about issues do not emerge from deep understanding”

It’s not really hard then to understand what (dafaq!) is happening around, especially in the current political scene in US and India.
Mass Psychology > Cognitive Dissonance > Confirmation Bias > Rise of Nationalism > Jingoism > Xenophobia >...Isolationism…

So what’s the alternative? Sloman and Fernbach don’t have a solution, and they’re well aware of the limits of their own understanding, and they know they don’t know the answer. In all likelihood, nobody knows...
If you like this book, you should probably club this with ‘The Enigma of Reason' by Hugo Mercier, Dan Sperber' show less

A bat and ball cost $1.10. The bat costs one dollar more than the ball. How much does the ball cost?

This question appears on Yale Marketing Professor Shane Frederick's CRT--"Cognitive Reflection Test." He found it in a book of riddles. (Collaborating!--he needn't create his own questions from scratch!) Those getting it and two other similar questions right are, we're told, in the minority. The CRT distinguishes people who like to reflect before they answer from those who just answer with the show more first thing that comes to mind.

I had no trouble with the bat and ball problem nor with the other two, not necessarily because I "like to reflect." But because I'd seen it before (as well as the other two!) In fact, I saw it most recently in The Enigma of Reason. And before that in Thinking, Fast and Slow. And before that in countless other places, probably as long ago as when I was still in high school. Maybe most of the people Prof Frederick tests hadn't seen it, but then again, maybe they did. People see terms like "Cognitive Reflection Test" and think they're dealing with scientific experts and believe what such people say. The prevalence of impressive jargon is one reason for what this book calls "The Cognitive Illusion" but not one getting much attention.

What I'm trying to say here is that even the authors of this book think they know a lot more than they do. They talk down to their readers and over explain the obvious. If the topic of this book and some of its observations weren't so important, I'd have given it a single star. The book often asserts "facts" that are poorly sourced or outright wrong. It appeals to the crowd of "most people" 45 times. "If you're like most people," you're this way or that way. Well, most people won't read this book. Nearly a third of Americans don't read books at all. Those who do will most of the time not choose this one. What's more, I'll bet (though I admit I'm just guessing here--something the authors should regularly remind us they often do as well) those who read this book are likely readers of the 2 books I list above with the bat and ball problem so who exactly is the audience this book addresses?

I'm willing to speculate that most of its readers are NOT like "most people;" that a good proportion will, for example, not see Christopher Columbus as a the hero the authors do. They will have heard of the Jonestown Massacre and the Dunning-Krugger effect and won't need them painstakingly explained. They will not think that the problem people have with compound interest is because "most people" think linearly but rather because there are large number of financial predators out there who make a good living out of legally deceiving people.

In addition, as another reviewer points out (see? I'm using team work in my review!) the collaborators who wrote this book don't work well enough together to give it the focus that one by a single author might. My guess is that there's a "most people" out there the authors would like to reach but will fail and end up talking to people more like me.

That's a shame because this book has a lot of important things to say. One of them is what Stolorow and Atwood are saying in The Myth of the Isolated Mind and another is what Barack Obama said in his "You didn't build that" speech. In brief, it's that many people forget that their confidence and knowledge and achievements are dependent upon a community of others.

Another point they make is that by downplaying our reliance on others, we become vulnerable to living in a bubble where there's no one to challenge our false beliefs.

A third important part of this book is their description of how science actually works (and so many other fields as well) which is not only rarely described but hidden as a vulnerability.

We give this book a star for each of these three topics, totaling 3 stars. Had they, for example, also spoken about the cult of individuality which is a religious belief in American society, and means that any ordinary attempt to reveal the collaborative nature of knowledge might be emotionally opposed as socialism, they could have gotten a 4th star, but this topic is never broached.

Instead they choose to theorize about measuring the intelligence of groups instead of individuals, "c" rather than "g" (they could have skipped the insider jargon and spoken of I.Q. and been less mysterious but they also want to create cognitive illusion when it suits their purposes). Here they also miss the opportunity to point out how many forms of discrimination in employment are hidden behind needing to hire people similar to the ones already hired so they'll work well together.
Their thesis requires that they pretend that individual intelligence is unimportant, but that isn't always the case. I suggest you read instead the more interesting discussion in Coders: The Making of a New Tribe and the Remaking of the World" of the programmers who can turn out 10 times the code of the average employee and the trade off between their outsized skill and the need for functioning and diverse teams.

So--a book which makes important points but ultimately fails to see the larger picture. show less

It all begins with toilets.
Everyone (throughout the developed world!) is familiar with toilets. A typical flush toílet has a ceramic bowl filled with water. When the handle is depressed, or the button pushed, the water—and everything that’s been deposited in it—gets sucked into a pipe and from there into the sewage system. But how does this actually happen?
In a study, graduate students were asked to rate their understanding of everyday devices, including toilets, zippers, and cylinder show more locks. They were then asked to write detailed, step-by-step explanations of how the devices work, and to rate their understanding again. Apparently, the effort revealed to the students their own ignorance, because their self-assessments dropped.
(Toilets, it turns out, are more complicated than they appear!)

Sloman and Fernbach see this effect, which they call the “illusion of explanatory depth,” just about everywhere.

People believe that they know way more than they actually do.

What allows us to persist in this belief is other people. In the case of my toilet, someone else designed it so that I can operate it easily. This is something humans are very good at. We’ve been relying on one another’s expertise ever since we figured out how to hunt together, which was probably a key development in our evolutionary history. So well do we collaborate, Sloman and Fernbach argue, that:
“We can hardly tell where our own understanding ends and others’ begins. One implication of the naturalness with which we divide cognitive labor is that there’s ‘no sharp boundary between one person’s ideas and knowledge’ and ‘those of other members’ of the group.”

So not just rationality but the very idea of individual thinking is a myth. Humans rarely think for themselves. Rather, we think in groups. We think we know a lot, even though individually we know very little, because we treat knowledge in the minds of others as if it were our own.
This is not necessarily bad. Our reliance on groupthink have us an edge over all other animals and turned us into the masters of (this) planet. The knowledge illusion enables us to go through life without being caught in an impossible effort to understand everything ourselves. From an evolutionary perspective, trusting in the knowledge of others has worked extremely well for humans.
This borderlessness, or, if you prefer, confusion, is also crucial to what we consider progress. As people invented new tools for new ways of living, they simultaneously created new realms of ignorance; if everyone had insisted on, say, mastering the principles of metal-working before picking up a knife, the Bronze Age wouldn’t have amounted to much. When it comes to new technologies, incomplete understanding is empowering.
But…the knowledge illusion certainly has its downside. The world is becoming ever more complex, and people fail to realize just how ignorant they are of what’s going on. Consequently, some who know next to nothing about meteorology or biology nevertheless conduct fierce debates about climate change (trump/trumpsters), while others hold extremely strong views about what should be done in Iraq or Ukraine without being able to locate them on a map. Also, It gets much more complicated in the political domain. How could we then vest authority in voters and customers who are so ignorant and susceptible to manipulation? If Sloman and Fernbach are correct, providing future voters and customers with more and better facts would hardly solve the problem. (Try using facts and proofs to convince half-witted, ignorant and imbecile Trump and Trumspters that climate change is actually a thing (and many other similar things that are established facts), and not a propaganda by China)
Encouraging people to be more realistic about their ignorance is, as it sounds, very hard!
People rarely appreciate their ignorance, because they lock themselves inside an echo chamber of like-minded friends, and self-confirming news-feeds, where their beliefs are constantly reinforced and seldom challenged.

“As a rule, strong feelings about issues do not emerge from deep understanding”

It’s not really hard then to understand what (dafaq!) is happening around, especially in the current political scene in US and India.
Mass Psychology > Cognitive Dissonance > Confirmation Bias > Rise of Nationalism > Jingoism > Xenophobia >...Isolationism…

So what’s the alternative? Sloman and Fernbach don’t have a solution, and they’re well aware of the limits of their own understanding, and they know they don’t know the answer. In all likelihood, nobody knows...
If you like this book, you should probably club this with ‘The Enigma of Reason' by Hugo Mercier, Dan Sperber' show less

I finally finished The Knowledge Illusion. The five stars here are given *despite* the writing: It annoyed me a lot, but the thoughts in the book are important enough that they changed my thinking on some issues, so five stars are appropriate.

That said: I liked a lot of the thoughts in it, and I disliked large parts of the writing (both the forced humour and the wandering repetitious examples). I think it would have benefited from strong editing. Despite my annoyance with the style and some show more of the discussion branches, I liked the core points they make very much.

The part where we know little about our surroundings is something I think about a lot since I can remember. Occasionally it turns into what I call "supply chain anxiety", when I start over-focusing on all the things I can't know, and it's been nice to read discussion of this issue. The part that was new for me concerned group intelligence, and the idea of focusing on group performance and intelligence over individuals. It's cool and makes sense, and I have a bunch of nonverbal intuitions on this topic that I'm hoping to flesh out in the future. show less