J. A. Kay

Fun and lively, but deep in understanding of the world. Deconstructs behavioural economics and nails overreliance on modelling, especially how most of the numbers are plucked out of the air. Wide arc of reference from Aristotle to Elon Musk, from the killing of bin Laden to the Edinburgh tramline.

This was probably the most thought-provoking book I've read this year.

When I say that, I don't mean it was the kind of book that made me reconsider everything I believed, though it did some of that. Rather, nearly every page had me pausing to consider and mentally debate the book's arguments, which were always interesting even when I ultimately came down against the authors. (Of course, I have a larger-than-normal interest in epistemology and uncertainty, so your mileage may vary...)

The show more main point of the authors if to respond to an intellectual movement that claims to be foregrounding uncertainty in their analysis. This movement, inspired by Bayesian statistics, tries to get away from narrative-driven beliefs by instead quantifying the likelihood of those beliefs, and constantly updating those likelihoods in response to new information. The authors argue this movement is misguided, and actually does the opposite of what its backers claim it does — by quantifying risk, they say, people are applying false precision to things that are actually radically uncertain — impossible to quantify.

"Reasonable uncertainty is uncertainty which can be removed by looking something up (I am uncertain which city is the capital of Pennsylvania) or which can be represented by a known probability distribution of outcomes (the spin of a roulette wheel). With radical uncertainty, however, there is no similar means of resolving the uncertainty — we simply do not know."

Put another way, it is the difference between "risk" and "uncertainty" where "risk" means "unknowns which could be describe with probabilities" and "uncertainty" which can't. Today, the authors argue, we tend to treat uncertain things as if they are actually risks that can be precisely quantified. They give the example of national security advisers meeting with President Barack Obama in 2011, giving their assessments of whether Osama bin Laden was actually in a compound in Abbottabad, Pakistan — one advisor said there was a 95 percent chance bin Laden was there, while another said 80 percent and another 40 percent. Obviously these percentages are completely different from, say, the 50 percent chance that a fair coin will come up heads.

But this example also brings up one of the problems with the book: it's rather over-focused on issues inside the field of economics (and adjacent areas), and the author's arguments against various forms of probabilistic reasoning run into more issues when they move past critiquing over-quantified economic models and move to day-to-day decision-making.

To return to the prior example, in a very literal sense, the statement that there was a 95 percent chance Bin Laden was in Abbottabad is meaningless. Either he was there or he wasn't; it wasn't like you could raid the compound 20 times and expect to find Bin Laden 19 times. But this wasn't a case where the only options for belief were "he's there," "he's not there," and "we don't know." One can believe it is "more likely than not" that something is true, that evidence suggests something but doesn't prove it. Saying "95 percent" may not have any solid statistical basis, but isn't it a perfectly fine synonym for "almost certain"? To be sure, we need to make sure not to take that 95 percent estimate too seriously, as a real, empirical probability. But at a certain point, applied to real life and not economic models, this argument becomes a straw man.

Another favorite straw man argument the authors use is to mock the idea that actual people making real decisions have a "Bayesian dial floating over their heads" — a reference to the model of Bayesian statistics, which starts with a "prior probability" that something is true and then updates that probability based on evidence. Real decisions, they say, are based on narratives, not statistical models. Again, this is an argument that is obviously true in a very narrow sense — as they demonstrate, even professional economists and statisticians usually don't use their probabilistic methods for making life decisions — but falls down a bit when taken a little more loosely. It's perfectly possible to approach life in a pseudo-Bayesian sense, starting with your belief about what is the case, and updating it as you learn more, even if you're not actually constantly performing Bayesian math in your head like an imaginary person in an economic model.

But even if many of their arguments fall apart a bit when applied to real life and not to economics, this is still a helpful book for lay readers. Their emphasis on knowing when to say "I do not know" and the value of asking "What is going on here?" are well-taken. And their targets aren't just straw men — over-quantified economic models are real, and used as the basis for all sorts of hugely consequential decisions. (Among other things, they cite the bank models before the housing bubble burst in 2007-8, for which the collapse of the housing market allegedly involved "25-standard deviation moves several days in a row." As they note, "our universe has not existed long enough for there to have been days on which 25 standard deviation events could occur"; the problem was the models' assumptions, inputs and algorithms were wrong, and considered an event that actually did happen as basically impossible.) I think their points are made too strongly for laypeople's purposes, and are too focused on economics rather than daily life, but they're still well-taken. show less

How did I like this book? Well, I'm not certain.

HA HA HA. It was OK, but it sure seemed to be saying the same thing over and over again for 433 pages and a lengthy appendix: Most situations in life involve a radical uncertainty, rendering them unsuitable for statistical modeling or probabilistic forecasting.

Of all the chapters basically saying the same thing from different perspectives, my favorite was "Evolution & Decision-Making," which leads with a quote attributed to SF author Bruce show more Sterling: "Computation is not thinking... You are much more like hour house cat than you are ever going to be like Siri."

To quote further the beginning of the chapter, "Behavioural economics has identiifed a raft of ways in which humans depart from axiomatic rationality. These behaviours are described as 'biases,' signs of human failure... It is as though God had given us two legs so that we could run or walk, but made on leg shorter than the other so that we could not run or walk very well... We are not defective versions of computers trained to optimise in small-world problems, but human beings with individual and collective intelligence evolved over millennia."

That's basically what it's about, although this is the only chapter with an evolutionary bent; it's all about how 'real-world' problems are nothing like 'small-world' problems that researchers come up with in the lab.

So take heart! You aren't a broken machine. You're an exceedingly smart house cat!

Also memorable, in the chapter "The Use & Misuse of Models," was a brief historical overview of the collapse of the cod industry in Newfoundland. "The [Dominion] Fisheries Office developed complex models on which its recommendations [for total allowable catch] were based. But cod stocks continued to decline. For the year 1992, the total allowable catch was set at 145,000 tonnes. That proved to be the last year of commercial cod fishing on the Grand Banks." The authors do not lay responsibility for this solely with the modelers, of course, but maintain that their 'evidence' ended up justifying the policy of "greedy fishermen and mendacious politicians" rather than actually protecting fish stocks. This example of mismanagement by model struck me enough to read up on the subject in Wikipedia, where the sad story can be read in more detail: "Over 35,000 fishermen and plant workers from over 400 coastal communities became unemployed... Newfoundland has since experienced a dramatic environmental, industrial, economic, and social restructuring, including considerable outward migration..."

Something of a detour from the main idea of the book; but one example of how the illustrations and anecdotes chosen by the authors are very powerful and well conveyed. Sorry to detour on the detour, but listen to a "No More Fish, No Fishermen," a song on this topic I heard long ago on public radio and never forgot:
https://www.youtube.com/watch?v=uPw74oTuliM show less

We live in a world where increasingly we believe that technology will answer our problems, and even our prayers. With sufficient effort, money and computing power, we can find the solution to any problem, and where we don’t it’s simply that we applied Insufficient effort, money and computing power and must try harder next time. The result is an unstoppable flow of instant solutions each attempting to address the problems created by its predecessor.

In this well-written and thoughtful book show more John Kay sets out why, in the real world, direct solutions are seldom the ones that work and how real change is approached not directly, but obliquely.

The book is divided into three sections. In the first Kay cites instances of oblique decision making in all aspects of our everyday lives. Section two explains why this is the case and how direct approaches are simply not possible. In section three he sets out guidance on how obliquity can be applied effectively.

This book is not against the application of scientific methods, nor an advocate for decision making based on intuition. Rather it recognises the true nature of the interconnected world and the impossibility of calculating the consequences of our actions in advance of those actions being made. It offers a well-reasoned argument for an alternative that will feel familiar to us in our everyday lives, but is rare in decision making elsewhere.

The following paragraphs from the book illustrate the approach:-

“In business, in politics and in our personal lives we do not solve problems directly. The objectives we manage are multiple, incommensurable and partly incompatible. The consequences of what we do depend on responses, both natural and human, that we cannot predict. The systems we try to manage are too complex for us to fully understand. We never have the information about the problem, or the future we face, that we might wish for.”

“But the idea that moral algebra is really the right way to make decisions, even if we eventually fail to use it, is deeply ingrained. So we tell ourselves that we are really using moral algebra when our real decision making process is oblique – we play Franklin’s Gambit. Franklin’s Gambit is perhaps the most common fault in decision making. – and particularly in public decision making – today. There is an appearance of describing objectives, evaluating options, reviewing evidence. But it is a sham. The objectives are dictated by the conclusions, the options presented so as to make the favoured course look attractive, the data selected to favour the desired result. Real alternatives are not assessed rigorously: policy-based evidence supplants evidence-based policy.”

This is a thought provoking book that I recommend to anyone trying to make sense of how decision making works or more often doesn’t work. show less