Snake Oil Science: The Truth about Complementary and Alternative Medicine

Occasionally, one comes across a book which really inspires and this is one of them. The author is a biostatistician (we called it Biometry when I studied it at University). And he has spent a considerable portion of his life investigating the impact of complementary and alternative medicines on our various ills and aliments. It more or less comes down to an investigation of the placebo effect but his great skill is in pointing to the difficulties in administering and interpreting double blind studies where the treatment involves some sort of procedure (rather than just taking a pill)....a procedure such as acupuncture or meditation. I must confess that I found his throughly scientific approach really stimulating and refreshing. But show more that also confirms my pre-existing biases so one has to proceed with caution. But Bausell DOES proceed with caution and seems to me to be exceptionally well balanced in all of his work and conclusions. For me it is a "tour de force". An exceptional work. Not only does he demonstrate that none of the various CAM procedures have an impact that is greater than the placebo effect; he also explains how the placebo effect occurs and the biochemistry behind (at least some of) it. I've extracted a few nuggets from the book as follows:
I believe the most enlightening aspects of my five-year exposure to the world of Complementary and Alternative Medicine (CAM) research revolved around my increased intellectual interest in a phenomenon known as the placebo effect. It is this phenomenon (which is at least as interesting and counterintuitive as any New Age health practice) that holds the key to answering this book's pivotal question: whether or not CAM therapies work......... And, while the placebo effect does not provide the complete answer to issues such as why so many people can be so sure that therapies such as homeopathy or acupuncture work for them (or why science is so impotent in supporting or refuting these beliefs), it does provide a starting point for their consideration. This is because the placebo effect is a prominent member of an extended family of logical, psychological, and physiological phenomena that conspire (separately and together) to confound our everyday thought processes.
We can examine the actual scientific studies that address four specific issues:
1. Is there such a thing as a placebo effect? In other words, can a completely bogus therapy work?
2. Is there something that has been demonstrated to take place within the body that could explain how a placebo effect occurs?
In other words, if there is evidence that a placebo effect exists, are these results consonant with findings from other scientific disciplines?
3. Is there such a thing as a CAM effect over and above what can be attributed to the placebo effect?
4. Is there something that has been demonstrated to take place within the body that could explain how one or more of these
 CAM effects occurs? Again, this addresses the issue of whether or not the existing evidence is consistent with what we know about the biology of the human body.
Our original question.....which is important enough to repeat: Is any complementary and alternative medical therapy more effective than a placebo?
We have entered an era of consumer dissatisfaction with conventional medicine's inability to treat, much less fix, chronic, sometimes disabling aches and pains. But, for better or worse, dissatisfaction tends to create demand, which in turn is met by supply. And in this case, what was being supplied was a truly bewildering variety of therapies, the vast majority of whose practitioners approached medical care from a holistic, nonbiological, non-pharmacological, non-invasive, non-evidence-based, non-scientific perspective.
.... in 2002, a second governmental survey of more than 30,000 people found that 36 percent of the American public had used some form of non-prayer-based CAM therapy during the past twelve months.
Let me first offer a very broad look at the field...... It includes:
(1) indigenous medical systems,
(2) recently developed (nonindigenous) medical systems,
(3) spiritual or energetic healing techniques,
(4) methods of relaxation, and
(5) extensions of conventional scientific findings.
And while some therapies arguably belong to more than one of these categories, their use provides us with a broad, succinct gestalt of the entire field.
In general most medical practitioners believe in the scientific process and value the evidence that it produces. In the face of definitive evidence, such as the recently conducted controlled trial that found knee surgery for osteoarthritis to be no more effective than a sham (placebo) surgical incision, most surgeons would be expected to eventually stop performing this particular operation. I seriously doubt, however, that there is a traditional Chinese medicine practitioner anywhere who ever stopped performing acupuncture on an afflicted body part in the presence of similarly definitive negative evidence.....CAM therapists simply do not value (and most, in my experience, do not understand) the scientific process.
With a few notable exceptions the placebo effect itself escaped serious scientific scrutiny until 1955, having largely been considered prior to that time to be more a part of medical lore (or physician mystique) than a documented clinical entity...... Beecher's research was relatively simple. He re-analyzed the results of fifteen clinical trials that had employed a placebo control group and concluded that 35 percent of the patients who had received a placebo responded positivel. The article became an instant classic,...... If patients participating in a clinical trial can improve simply because they believe they are receiving an effective medical intervention, how can anyone have any confidence in the results of any clinical trial that did not employ a placebo control group?.... Called "double blinding," this strategy soon became a scientific requirement for serious clinical trials and has since been employed for almost all types of medical interventions. Of course, creating credible placebo control groups is a lot easier in investigations of drug interventions..... than is the case for other types of therapy.
The bottom line for us here is that the placebo effect, in conjunction with factors such as the natural history of our symptoms, conspires to produce false positive conclusions.
Psychological factors impeding our ability to draw correct causal inferences
A reluctance to admit when we are wrong (cognitive dissonance).?
Simple optimism (possessing an internal locus of control).
Dr. Smith's continued belief in his CAM therapy of choice really is quite comparable to Sarah's continued acceptance of Prevention's recommendations of CAM therapies and in some ways is more understandable, He, after all, was forced to rely upon his patients' truthfulness, plus he had many patients to monitor and he saw them only periodically. Even more problematic, physicians' judgments regarding their successes and failures have a serious built-in bias: patients who do not believe they are being helped tend not to come.
Test subjects who guess they are in a placebo group and leave the trial will bias the results. So randomized, placebo-controlled trials can be worse than useless unless they are conducted very, very carefully under the best of circumstances. And evaluating a CAM therapy that involves a procedure, as opposed to a pill, is far from the best of circumstances due to the likelihood of experimental bias and unpersuasive placebo controls, among other things.
In a perfect world it shouldn't even be necessary to explain why something works if we're sure that indeed it does. After all, we knew that penicillin was a true wonder drug years before we knew what happened in the body to make it so effective.
One week we hear that oat bran reduces serum cholesterol; the next report we see says that it does not. First we are told that estrogen replacement protects against heart disease; later we are informed that it causes it.?
And as bad as this situation is in conventional medicine, it is far worse in CAM. There are many CAM therapies that have never been evaluated via a randomized, placebo-controlled trial. And there are therapies that have been evaluated multiple times so poorly... It has been estimated, in fact, that more than 500 RCTs (Randomised Controlled Tests) have been conducted to evaluate acupuncture alone, half of which have been placebo-controlled, yet the number of high-quality acupuncture trials could probably be counted on one's fingers.
All CAM experiments are not created equally, as illustrated by the following hierarchy:
1. Randomized clinical trials (RCTs) are more credible than nonrandomized ones.
2. Large RCTs (those with at least fifty patients per group and preferably more than one hundred) are more credible than small ones.
3. Large double-blinded RCTs employing placebo control groups are more credible than RCTs not employing placebos, especially for CAM trials.
4. Large double-blinded randomized, placebo-controlled trials with relatively minor experimental attrition (preferably less than 20 percent but certainly less than 25 percent) are more credible than large randomized, placebo-controlled trials with high experimental attrition.
5. Large double-blinded randomized, placebo-controlled clinical trials with low attrition rates published in high-quality journals are more credible .... than large randomized, placebo-controlled clinical trials published in, say, complementary and alternative journals that consistently have a bias favouring studies showing that CAM works,
Research has provided a definitive answer to the question of whether acupuncture is capable of relieving acute dental pain (in this case pain following dental surgery) over and above what can be attributed to the placebo effect. Thanks to Dr. Lixing Lao, I was able to participate in producing this answer via a pair of randomized, placebo-controlled trials...... The results were quite definitive: there was no statistically significant difference between the real acupuncture group and the two placebo groups with respect to the average pain experienced following dental surgery. But Dr Lao then conducted a second, larger study... While it is undoubtedly immodest of me to say this, it should be noted that these two trials were of extremely high quality for CAM re-search...... Lao also did something that all investigators (CAM and conventional) should do but seldom bother with: he instituted a check to see if his placebo groups really did indeed trick people.
Question:
Which treatment do you think you received?
1. Real acupuncture?
2. Fake acupuncture?
3. Not sure?
There was virtually no difference between the placebo group and the treatment group but a significant difference between those who believed they were getting treatment and those who thought they were getting a placebo. In the final analysis, it doesn't much matter what the real" explanation for these results is since we know that patients didn't make their guesses based upon the effects of acupuncture. Why? Because there weren't any effects for acupuncture. The participants made their guesses based upon their personal beliefs and expectations..... This relationship can be summed up as follows: If we believe in CAM therapies, then they will most likely work for us. If they don't work for us, then we will find a reason for this failure and continue to believe in them-thereby prohibiting reality from conflicting with our beliefs.
Is there really such a thing as an analgesic placebo effect?
Even the best of us (biostatisticians) can't be sure that what is reported was actually done, that all of the study limitations (and mistakes) have been reported, whether the investigators allowed their biases to influence their results, and— most important from a bottom-line perspective-whether the results reported are valid...... So how do we get around these problems? By requiring the independent confirmation of important findings.
There is a considerable amount of research that demonstrates that humans can be conditioned, just as Pavlov's successors did with dogs, to respond to placebos through repeated administrations of active drugs. In one study using healthy women, for example, two researchers administered nitroglycerin in distinctively flavored tablets over the course of several weeks. Then, following the administration of the same flavored pill without nitroglycerin (Le., a placebo pill), a change in heart rate similar to (but less dramatic than) the response for nitroglycerin was observed.3
Similarly, humans apparently will respond physiologically to placebos they think are caffeine, nicotine, alcohol, and a wide variety of drugs (e.g.. interferon, bronchodilators and bronchoconstrictors, stimulants and sedatives, and chemotherapeutic agents) via conditioning based upon their past experiences with those substances.*
The conditioned responses to the placebo drugs, while substantial, are almost never as strong as the initial responses to the real drug itself.
There is some evidence that people's desire (which can be conceptualized as motivation or perceived need for pain relief also contributes to the magnitude of the placebo effect.
There is also an intriguing genre of research suggesting that the desire or motivation to improve one's health may be an important factor in eliciting placebo effects..... patients who conscientiously take their placebo tablets tend to improve more than those who don't, even though there is absolutely no medical reason why this should happen.
Personally, based upon experimental evidence, I would be shocked if there weren't a placebo effect in some of the drug treatments for depression and Parkinson's disease!' More-over, some evidence exists (although it is based on findings that haven't been replicated) to show that placebos affect postoperative swelling, movement disorders, vital signs such as oral temperature and pulse....
Donald Price and Henry Fields concluded from a set of studies that there are three primary factors involved in placebo analgesia:
1. classical conditioning effect that occurs without the subject's conscious awareness of the conditioned stimulus-unconditioned stimulus association;
2. a desire for a given treatment or agent to significantly relieve pain; and
3. the level of expectation that pain will be significantly relieved by such treatment or agent. Although we think that classical conditioning is a major determinant of the magnitude of the placebo effect, we propose that a combination of desire and expectation can be of equal if not greater importance?
Principle 1: The placebo effect is real and is capable of exerting at least a temporary pain reduction effect. It occurs only in the presence of the belief that an intervention (or therapy) is capable of exerting this effect. This belief can be instilled through classical conditioning, or simply by the suggestion of a respected individual that this intervention (or therapy) can reduce pain.
We have all been conditioned to expect our physicians to help us based upon the fact that they really have helped us in the past. (Or at least we think they did, based upon natural history and the other inferential artifacts discussed earlier.) We have, in other words, been conditioned, à la Pavlov's dogs, to expect the medications they give us to work. Second, these findings explain how CAM therapists could inadvertently trick their patients into believing that their pain has been relieved. And then, based upon their patients response, these very real placebo effects could reinforce the CAM therapists' belief in the effectiveness of their therapies.
Price and associates found that the memory of the pain relief afforded by the placebo "treatments" was considerably more dramatic than the pain relief the participants actually experienced at the time. This means that if CAM therapies really are placebos, their recipients will remember them as being more effective than they actually were.
Principle 2: The placebo effect has a plausible, biochemical mechanism of action (at least for pain reduction), and that mechanism of action is the body’s endogenous opioid system.
Andrew Vickers and his colleagues set out to ascertain if the nationality of the principal authors influenced the results of the trials they undertook..... What I think these results demonstrate very definitively is that CAM investigators' countries of origin must be considered in the interpretation of CAM effectiveness trials. Chinese-speaking countries, for example, simply did not produce anything but positive acupuncture trials, while parts of Europe did not lag that far behind. And when Vickers and his colleagues repeated the analyses with a much larger sample of trials involving treatments other than acupuncture (most of which involved conventional medical treatments), they basically came up with the same results: 98 percent of conventional Chinese trials produced positive results, as did 97 percent of Russian trials. Reports from Canada, Australia and NZ averaged 30% positive responses to acupuncture; US Studies 53%, Scandinavia 55%, UK 60%, Other European 78%, Asia 98%.
There is another source of bias.... publication bias, and it refers to a well documented tendency for research journals to favour positive results (e.g., acupuncture helps arthritis sufferers) over negative results (acupuncture doesn't help arthritis sufferers) when deciding which articles they will publish.
For every twenty randomized, controlled trials conducted, one will be blessed as statistically significant when the differences between the experimental group (CAM therapy, in the present context) and the control group (or placebo here) in fact occurred by chance alone and were not real. What this means, in turn, given our current task of coming up with a definitive answer to the question of whether or not CAM therapies work better than a placebo, is that 5 percent of those trials judged to be positive really weren't.....
it certainly means that we have to interpret the totality of the research in any given area with extreme caution. It also means that we must always keep these biases (due to the publication system, experimenters, scientific acculturation differences, and the 5 percent error rate associated with statistical significance) in mind. And it is even more important to realize that all of these factors, not to mention the legions of inferential artifacts already discussed, conspire to produce false positive results.
What kind of CAM research should we consider acceptable to evaluate? Preferably:
High-quality, large (involving more than one research site if possible, which helps reduce experimenter bias), placebo-controlled RCTs verified by independent investigators (which is far and away our best protection against bias).
Our results so far indicate that when high-quality, placebo-controlled trials in high-quality, selective journals are considered, the preponderance of the evidence suggests that CAM therapies do not produce beneficial effects over and above those that can be explained by the placebo effect......But the bottom line is that there just isn’t enough of this sort of evidence to arrive at a truly convincing conclusion about this question.
In terms of reviews of published work, small positive trials begin to predominate, are not counterbalanced by equally small but unpublished negative trials, and can easily overwhelm the effect of a larger, high-quality trial in a systematic review. And in the Cochrane CAM reviews, small trials (often published a couple of decades before the CONSORT statement was formulated) are the rule rather than the exception.
Bausell has attempted to objectively classify the ninety-eight systematic reviews he located as providing either a positive (+) or negative (-) and goes through all the 98 studies with comments and a plus or minus and came up with 21 reviews that found positive impacts but then discounts a lot of them for various reasons (including later better studies that showed negative results) and ends up with a figure of 5% which, as he points out, is the figure which Ronald Fisher said would show up as false positives even where there was no real effect.
So is there sufficient evidence to conclude that any CAM therapies are more effective than a placebo? Based upon the evidence presented in both the previous chapter and this one, the bottom-line answer is no.
And this conclusion will now be elevated to the status of the book's third principle:
Principle 3: There is no compelling, credible scientific evidence to suggest that any CAM therapy benefits any medical condition or reduces any medical symptom (pain or otherwise) better than a placebo.
Unfortunately, there is practically no plausible evidence supporting any biochemical mechanisms for CAM therapies. What we'll have to do, therefore, is to concentrate upon the hypothesized biological mechanisms of action proposed for CAM therapeutic effects by their proponents.
He briefly treats a range of different alternative systems: Traditional Chinese medicine, ayuvedic medicine, Tibetan medicine, osteopathic medicine, homeopathy, and some individual therapies; yoga herbalism, chiropractic, spiritual, chelation therapy, hydrotherapy, meditiation/mindfulness, massage, magnetic therapy et. Etc.
So what's the bottom line? I think that most of us can at least agree that what we have here is a wide assortment of extremely creative therapies with even more diverse and creative hypothesized mechanisms of action. In some cases, these therapies ostensibly work by accessing energies, physiological pathways, and/or dimensions of existence that have not yet been observed, documented, or measured. In some cases we have therapies that borrow mechanisms of action from scientifically verifiable phenomena but stretch them far beyond anything recognizable by conventional science...... In no cases, however, do we have anything that would survive William of Occam's parsimony principle if we use the placebo effect as the comparator...... the one thing their use shares in common with the sole triggering mechanism for the placebo effect is belief.
All of which leads to our fourth and final principle:
Principle 4: No CAM therapy has a scientifically plausible biochemical mechanism of action over and above those proposed for the placebo effect.
Which regretfully leads me to conclude that: CAM therapies are nothing more than cleverly packaged placebos.
And that is almost all there is to say about the science of CAM.
CAM believers number well above 50 percent of the population, so the discipline may never fall to "oblivion" and its positive effects can be easily reproduced as long as a credible placebo control group isn't employed.
Based upon [my] admittedly limited experiences, it is my opinion that many reporters (scientific or otherwise) have not received a very good scientific education and have an absolute aversion to dealing with statistical concepts. Which probably lead to the three characteristics of the press' coverage of science:
1. "Superficiality is easier [to present] than depth" (and people who have something to sell have an exceedingly simple agenda to get across).
2. "The media cannot deal with ambiguity, subtlety, and diversity" (which always characterizes scientific endeavours involving new areas of investigation or human behaviour in general).
3. "The bizarre always gets more attention than the usual" (and things don't get much more bizarre than some CAM therapies)
It has been said, in fact, that prior to the development of antibiotics, about the only effective medicines physicians had in their bags were aspirin, laxatives, and placebos!
One thing I have saved you is the futile experience of going online to find out what does and does not work, which is the electronic equivalent of consulting a Ouija board. What you'll find there in the way of research evidence is a plethora of breathlessly positive studies conducted primarily by (or selected for presentation by) CAM proponents.
As mentioned above. I really liked this book. Have already recommended it to a couple of my academic statistical friends. Five stars from me. show less

So-called "complementary and alternative" medicine (or CAM) is a big business these days, and a great many people, some of them doctors and scientists, are thoroughly convinced that these unconventional treatments really work. But do such things as acupuncture and homeopathy truly treat anything, or are they just placebos with good PR? R. Barker Bausell attempts to answer this question.

Actually, although the focus here is on CAM, I think the usefulness of this book is much broader than that: it's a good, thorough, detailed look at what it takes to determine with any reasonable degree of confidence whether something has a real, non-placebo-based medical effect or not. And that's not nearly as easy as it looks. There are a whole host of show more factors that can make it seem, or even make it seem obvious that something is working when it's not. And not all scientific studies are created equal when it comes to controlling for those factors. Bausell explains the hows and whys of all this clearly and in depth, and applies it towards an evaluation of various CAM fields.

There are a few things here I'm inclined to quibble with, notably his disturbingly off-hand dismissal of non-English-language studies and his suggestion that people with chronic problems might as well go out and get some scientifically unsupported CAM-based treatments, anyway, for the power of the placebo effect. I also think he does best when he concentrates on specific treatments, such as acupuncture, as CAM is almost too broad a subject to take on all at once. Still, overall it's a very worthwhile read, and provides some excellent lessons about how science works in medicine, how good science and sloppy science differ, and why that difference is so important. show less

R. Barker Bausell sets out to write a pithy account of how he and his team debunked acupuncture. As a good scientist, he builds his case carefully, explaining for the uninitiated how the scientific method is supposed to work and going into great detail in describing the placebo effect. This is interesting stuff and may come as something of a revelation to non-scientists. Unfortunately, having demonstrated his case with respect to one type of CAM (complementary and alternative medicine), he uses it as the basis for generalizing to anything that might be classified as CAM. Here we have a fine scientist indulging in some grave errors of logic, and so undermining his own credibility. In the last third of "Snake Oil Science", Bausell show more flippantly dismisses anything he labels CAM, including some treatments that arguably are not. He does this not through an appeal to fact but by resorting to fallacies that most careful thinkers have learned to avoid. First, Bausell defines CAM circularly: a CAM technique is anything which is practiced by CAM practitioners. Then, having earlier shown that many CAM proponents justify their practice with faulty research, Bausell argues that any research produced by CAM practitioners is therefore faulty and discredited. Erroneously affirming the consequent in this way, instead of taking the time and trouble to build a logical case against these other forms of CAM, shows a glib and facile side to this work that serves to cast more doubt on the author than on the CAM therapies he purports to refute.

Many critical thinkers, scientists, and clinical researchers are eager for well written, logically and scientifically rigorous refutations of CAM, which is viewed as a useless drain on the health care system and a source of false hope to sick people. This book makes a good start, but sadly finishes badly by playing a game of "gotcha" that doesn't quite work out. show less

Subtitled “The Truth about Complementary and Alternative Medicine”

In short, the truth is all the effects can be explained by the placebo effect, and there are negative results universally when well controlled and blinded studies are done. The author is a bio-statistician, writes amusingly, and takes pains to define all current major alternative therapies. He recounts his own experiences working with the office of complementary and alternative medicine at NIH, designing well constructed trials. He writes on page 3 “We have, in short, entered an era of consumer dissatisfaction with conventional medicine’s inability to treat, much less fix, chronic, sometimes disabling aches and pains. But, for better or worse, dissatisfaction show more tends to create demand, which is in turn met by supply. And in this case, what was being supplied is a truly bewildering variety of therapies, the vast majority of whose practitioners approached medical care from a holistic, nonbiological, nonpharmacological, noninvasive, non-evidence-based, non-scientific perspective”. The text exhaustively and usefully reviews sources of bias in experimental results, and a long section reports on all the double blinded studies the author could find of alternative medicine, with virtually no positive studies. A very useful book. show less

Very readable and engaging. However, on this subject, Edzard Ernst's book Trick or Treatment is much more authoritative (and equally fun to read). Ernst has iron-clad credentials, to boot, and is far more sympathetic to CAM than Bausell. Yet Ernst is first and foremost in search of what helps his patients, and to do that he must truly seek the truth about CAM. In that, he excels.