AI Snake Oil: What Artificial Intelligence Can Do, What It Can’t, and How to Tell the Difference

I was unable to finish this book. I found the topic timely and interesting, but there were too many things in the book that made me wonder how well-edited it is. While I do agree that the two authors are experts in the field, there were a couple of inaccuracies for a book published by academics with PhDs (one was a PhD candidate at the time of publication) and published by a university press that has a peer review process.

The first thing that bothered me was a reference to how AI is used to predict hospital needs for patients enrolled in Medicare, and how an outcome has been elderly hospital patients denied care that they need based on these predictive models. I instantly raised an eyebrow as I read about this and thought to myself "I show more bet they mean Medicare Advantage." And sure enough, when I turned to the reference list at the end of the book, the title of the article that was referenced is "Denied by AI: How Medicare Advantage Plans Use Algorithms to Cut Off Care for Seniors in Need."

I thought maybe the two authors didn't live in the US, so they didn't understand the difference between Medicare (a government-run, taxpayer funded service for providing healthcare to seniors and people with disabilities) and Medicare Advantage (a private market health insurance plan marketed to the same demographic). However, the authors definitely mention that they live in the US on more than one occasion, so then I thought, maybe they just don't know the difference. But then, why are they writing about this topic as if they do?

I was willing to give that a pass, but later in the book I got to a blurb on weather forecasting that was either very poorly written or just outright wrong. Here's the blurb: "To be pedantic, the mathematical definition of calibrated weather forecasts is that out of all the days on which the forecaster said there is an x percent chance of rain, it rained on x percent of the days."

That is the most tortured, circular reasoning, reverse-logic statement that I've ever read. I know that weather forecasting is complicated. I know that "40% chance of rain" isn't as easy as "there is literally a 40% chance it will rain today, so you may not want to pack an umbrella." Forecasting has to do with complicated modeling, creating a prediction, as well as considering the geographical area that the model covers (as in: will it rain everywhere in Cook County, or just in a small portion of it?). And yes, modeling algorithms do come from historical data as well as current physical knowledge about thermodynamics and the atmosphere. I've done a small amount of research, and get the impression that forecasting calibration does occur, and it appears to be a way of reducing bias in a forecast. Most of the results on my search appear to be proprietary, so they don't share exactly how they calibrate data. I'm could not find a public resource such as a .edu meteorology department website explanation or a Wikipedia page to explain the process to me. In addition, the authors provided NO SOURCE for this quote whatsoever. Which made the "[t]o be pedantic" even more insulting to me. If you're going to be pedantic, tell me what source you used to write this quippy little footnote.

However, based on my own ability to scientifically reason, I can tell you that it makes no sense for a forecast to be changed to be consistent with the percentage of days that it rained in the past. A forecast cannot be changed to be consistent with historical data until we know whether or not it rained. In which case, it would no longer be a forecast.

I actually was so annoyed after having read that definition (and spent precious mental energy trying to make sense of it) that I yelled out loud, multiple times, that it made no sense, before finally calming myself down and turning the page. OK, I have high standards, but authors aren't perfect. I told myself to get over it and move on.

I turn the page, and lo and behold, there is a photograph of an Enigma machine. It is labeled Figure 5.1 (a). The next page has a photograph of a reproduction of the Antikythera mechanism, labeled Figure 5.1 (b). The following page has a photograph of a Hollerith tabulating machine, labeled Figure 5.1 (c). I am very interested in computing history, so I can identify these machines by looking at them. Then I read the caption, which states that (a) is the Antikythera mechanism, (b) is a Hollerith machine, and (c) is an Enigma machine.

That broke me. I'm sorry, but Princeton University Press's website tells me that a peer review and editorial review process occurs. Was this book held to lower standards because its intended audience is laypeople and not academics?

I've written books before. I understand that mistakes get made unintentionally. Honestly, the mis-captioning of the figures would have been a minor annoyance that I would have absolutely forgiven, if it hadn't followed other head-scratching moments. At a certain point, regardless of how much I want to read a book, I can no longer trust the authors to give me knowledge without me having to spend a ton of time fact-checking them. I don't have the time to both read this book AND go through the citations to ensure they're accurate. show less

A clear-eyed examination of the field of AI which points out both the pros and the cons. Well organized and easy to follow for neophytes