The Instability of Truth: Brainwashing, Mind Control, and Hyper-Persuasion

Lemov is a Harvard historian of science focusing on the 20th century psychology of control, i.e. brainwashing. This book promises on the cover to have some interesting insights on what Lemov deems "hyper-persuasion", the algorithmic social media driven manipulation of emotions and worldview. Unfortunately, it is very much a case of finding two trees and declaring a forest, and while the historical case studies on Korean War POWs and MKULTRA, and then the cult/anti-cult debates of the 70s and 80s, are fascinating, anything applicable to the 21st century is confused mess.

First my credentials. I have a PhD in science and technology studies focusing on the social aspects of ADHD. These days I'm a data scientist for a large and malevolent show more social media company. And in my free time I'm a Cold War history buff. I won't claim I'm particularly hot shit at any of these topics, certainly not to the level of Harvard Professor, but this book lands squarely in the intersection of things I'm an expert in. And as an apology, I listened to this one on audio, so I don't remember specific names and I don't care enough to go back and find out.

It's impossible to discuss brain-washing without the Korean War. The exact phrase slightly predates hostilities, coming from a CIA agent collating reports out of Hong Kong, who translated "brain-washing" out of a technique the Chinese Communist Party called "The Method", a comprehensive program of development based around personal confession, ideological reinforcement, and collective testimony. The Method was deployed on people from Mao on down, and seems to me a fairly standard indoctrination technique, if one with a bit more development and discipline than usual.

However, when the method was applied to American prisoners-of-war, the effects were enough to prompt panic at the highest levels of government. A number of captured pilots made statements saying they had used biological weapons on North Korean. Thousands of returnees seemed dazed and different. 21 captured soldiers didn't return at all, defecting to the Communist side. It appeared that the Communists had a weapon capable of breaking minds, a psychological warfare killshot.

A measured evaluation would suggest that this was not true. Almost all of the effects could be explained by mundane torture and deprivation. The 21 defectors were a sign of the arrogance and weakness of American political education, which just assumed that anyone graduating boot camp was a patriot. Of course, measured evaluations were not in the style of the Cold War. One outgrowth was the rise of SERE (Survival Evasion Resistance Escape) training for service members facing capture, where pilots and soldiers were preemptively tortured to help them understand what might happen. SERE techniques were then applied as actual torture to "enemy combatants" captured during the War on Terror.

The second side was MKULTRA, a CIA program that did everything from covertly dosing people with LSD to supporting a host of psychological research programs. MKULTRA affiliated researchers worked in the Haight-Ashbury during the Summer of Love, and tried to start up the Violence Lab at UCLA to uncover the psychological and social causes of violence. One unfortunate Boston engineer had his amygdala burnt out with electrical surgery, turning him into an insane derelict.

This would have mostly been another CIA embarrassment to be swept under the rug, except that cults became big deal in the 1970s. Starting with the Manson Family trial and then Patty Hearst's kidnapping and affiliation with the Symbionese Liberation Army, to the hundreds of thousands of people who joined less murderous "High Demand" religions, figuring out what happened to people in these circumstances and how to get them out became a big deal. Berkeley psychologist Margaret Singer is a focus of this chapter, from her work as an anti-cult deprogrammer and expert witness, to her reputational collapse. As Lemov puts it, major cults such as Scientology and the Moonies funded academic conferences targeting sociologists of religion, framing themselves as benign "new religious movements." The behavior of anti-cult deprogrammers didn't do them any favors, with standard tactics of kidnapping cult members and isolating and berating them basically identical to cult practices. The inability to firmly nail down what brain-washing was in a scientific sense eventually rendered it a legally useless defense.

With the benefit of distance, Lemov has a clearly definition of brain-washing. First, it's based in a traumatic rupture, a psychological frame simply not available to experts prior to the standardization of Post-Traumatic Stress Disorder in the early 1980s. Second, brain-washing is a primarily social phenomenon, focused around conformity to the norms of a new group. However, as cognitive and emotional creatures, its hard to impossible for humans to behave convincingly without also coming to believe accordingly. Victims of brain-washing experience an "agentic shift", where a new autonomous personality follows the norms of the cult, while a suppressed hidden personality voices inner doubts.

These two case studies are handled well. Unfortunately the book falls apart in the last third, where Lemov tries to apply this to the 21st century. The thing about conventional brain-washing is that it's incredibly labor intensive. Could digital technologies provide a way for this to happen at scale? Lemov closely examines the 2012 Facebook mood altering study, where Facebook data scientists showed almost 700,000 unwitting users either more negative or positive content on their feeds and tracked their subsequent posts. Those shown negative content posted more negative content themselves, as measured by sentiment analysis. The results were statistically significant thanks to the large sample, but the effect was miniscule, a 0.01% change (and I can say that as an employee at Big Malevolent Tech, that change wouldn't count by our own metrics). She then wanders though a rather glossy take on LLM chatbots, especially Replika, South Pacific cargo cults as a rational response to much higher levels of technology, and how an astrocrypto influencer, who tells people how to use astrology to invest in Bitcoin, is a basically kind and benign figure.

This book came out in 2025. Attitudes towards chatbots have changed a lot over the past year, with the various ChatGPT psychosis/suicide lawsuits, and I don't know when Lemov sent book_FINAL_v3_final_this_time_for_real(3).docx to the publishers, but the fact is that the last third feels confused and incoherent by the standards of 2021 on online misinformation, algorithmically driven extremism, and related topics, let alone for something contemporary. Maybe someone told her that a book on the 20th century history of psychology wouldn't sell and she had to add something relevant, or maybe she genuinely cares but is just out of her scholarly depth. Either way, the last third is simply embarrassing.

One major question which Lemov opens but does not follow up on are the differences between brain-washing and education. She describes feeling herself "colonized" by Lacan and Deleuze as an early grad student, and then consciously rejecting the obscurism of the poststructuralists. Education is (ideally) self-willed and beneficial, but again, I have an STS PhD from a place that takes the Strong Programme seriously. You can't just say that science is true because it reflects nature and psuedo-science is wrong because it doesn't. The same approaches and evidence must be applied to both. Belief modification is belief modification, so how can some kinds be education and some kinds be brain-washing? I think that she's right to point to trauma as key, and that approaching and/or creating traumatized individuals is what makes brain-washing distinct.

While the history is engaging, the broader implications (as we say in the business) are a clear miss. show less

2025 book #37. 2025. The author weaves and interesting tread from Korean War POWs, CIA mind experiments in the '60s, cults in the '70's, social media and crypto currency and how we may be effectively mind controlled with even realizing it happening. A good read.