The Master Algorithm: How the Quest for the Ultimate Learning Machine Will Remake Our World

It's taken me forever to finish this book, which I originally picked up in 2022 to learn more about machine learning. While it does a good job of going through the different angles and techniques for approaching machine learning problems, I got fed up with the "master algorithm" discourse very early on, and that was one of the reasons I didn't feel like reading more.

I was looking for a book that explains ML and AI concepts for non-practitioners. It did a great job at that. It goes through the most used 7 types of ML algorithms/concepts and explains how they work using high-level math and analogies. It blends a bit of the history of the field so that's always nice to contextualize the information.

What made me only give it 3 stars are the detours it keeps making into predicting the future and the crazy soft stance it takes towards the tech giants like FB and GOOG. I'm not sure if Mr. Domingos is still willing to be have such a friendly opinion of them after the latest findings on how they use ML and the many privacy breaches they've had, but I hope not. In any case I'm judging the books version of show more events and it's lacking any criticism towards the uses of ML in manipulating public opinion.

The book is great at teaching you the high-level workings of ML. The problem I had was it kept trying to do more than that by taking a stab at predicting the future of the field influence on humanity. Even for a specialist, it's mostly speculation. show less

This is a great book about machine learning for both: people who want to know more about it and people who are in this business. Domingos is a great writer with the ability to present complex ideas in easily digestible form. Here you will learn not only what you can do today with machine learning, but also the various philosophical camps, their approaches and finally the limitations of machine learning.

What is interesting about machine learning is that all algorithms in use today are statistical methods developed primarily in the 1960s and 70s. The only negative thing I have to say about this book is that I find some of his optimism a bit unfounded given the limitations of all of these techniques.

Quite hard to understand. It's a good read, nonetheless. It gave me an idea of how machine learning works, what the basic algorithms are. Parts of the book are super incomprehensible

Machine Learning Made Easier (or NOT!): "The Master Algorithm" by Pedro Domingos Published September 22nd 2015.
 
 
How can one become an expert in ML? All one needs is a basic background in (multivariate) Calculus, Linear Algebra, and Probability. ML is math. If one wants to understand the techniques, one has to understand the math. No shortcut. If one wants to start looking into the field of ML, this book is for you. If not, stay well clear.
 
My background is in computer science and software engineering and I've been interested in ML since I can remember. In 2013 I took Andrew NG's ML class at Stanford University (for those of you who want to dive into stuff like this here are mynotes of the class; while learning the needed math can show more look daunting at first it is actually quite fun once you get into it), and I was never literally the same…After that I made some Python coding to get a feel for the real thing, which I’m still doing to this day.
 
Humans ARE machines, albeit biologically-based. Billions of highly interconnected neurons receiving sensory input, lots of internal feedback, and signals that go out to motors, etc. Emotions, feelings, consciousness, are all just “concepts” we've constructed through a mixture of self-introspection and communicating with other self-introspecting machines (humans).
 
Read on, if learning comes as second nature to you. show less

Machine learning can solve important problems by looking at data and then finding an algorithm to explain it.
The review is really just based on a Blinkist summary version and so it’s a little unfair to the original author. But I’m very impressed with what the author is saying that I’ve given the book five stars. Well worth reading...and I might get around to the original some day.Here are some nuggets that I identified in the Blinkist version.
Unlike recipes, algorithms are sequences of precise instructions that produce the same result every time.....These standard algorithms are designed to accept information as an input, then perform a task and produce an output......But machine learning, or ML, algorithms are one step more show more abstract: they are algorithms that output other algorithms!......This comes in handy for finding algorithms for tasks that human programmers can’t precisely describe, such as reading someone’s handwriting.
Thanks to machine learning, we don’t have to. We just give a machine learning algorithm lots of examples of handwritten text as input, and the meaning of the text as the desired output. The result will be an algorithm that can transform one into the other. Once learned, we can then use that algorithm whenever we want to automatically decipher handwriting. And, indeed, that’s how the post office is able to read the zip code you write down on your packages.
What’s great is that ML algorithms like this one can be used for many different tasks, and solving emergent problems is only a matter of collecting enough data......For example, you might think that making a medical diagnosis, filtering spam from your email and figuring out the best chess move might all need completely different algorithms. But, actually, with one ML algorithm and the right kind of data, you can solve all these problems. [Though, one issue appears to be that generally we don't understand what the algorithm is or how it was derived...and maybe the computer found a one to one correspondence with lkx
To avoid hallucinating patterns, learning algorithms need to be restricted and tested for validity.
To hallucinate is to see something that isn’t really there. Interestingly enough, hallucinations are a central problem in the world of algorithms. In 1998, a best-selling book, The Bible Code, claimed that the Bible contained hidden predictions that were revealed by selectively skipping certain lines and letters. Critics disproved this assertion, however, by demonstrating that similar “patterns” could be found in Moby Dick and within Supreme Court rulings.
This is a good example of hallucinating patterns, which, in ML lingo, is the result of overfitting......When you throw enough computing power at a data set like the Bible, you will always find patterns because the computer can construct increasingly complex models until some arise. So, to get your algorithms under control, their power needs to be bounded by limiting their complexity....When you are preparing your original data set for the learning algorithm, it is important to divide it into a training set, which the algorithm uses to learn, and a holdout set, against which to test it. This way you can double-check the results. Ensuring the validity of the results is what an ML expert’s work is all about.
Rules using deductive reasoning and decision trees can allow machines and algorithms to think logically.
The world of machine learning has specialized branches with their own perspectives and preferred style of algorithms.
Symbolists, for example, manipulate symbols and learn rules in order to create artificial intelligence (AI).....The symbolists’ preferred algorithm is inverse deduction.
Generally speaking, inverse deduction creates rules by linking separate statements, like the following: “Napoleon is human” and “Therefore Napoleon is mortal,” the algorithm can arrive at broader statements, such as “Humans are mortal.”
While this kind of algorithm is good for data mining and sorting through relatively large amounts of data, such as medical records, it is costly and inefficient for truly massive databases. To make this work less complex, you can use decision trees to find these rules.
For example, if you wanted to come up with rules for sifting through medical records, you could use a decision tree. You’d start out with all of the records, but then, at the various branching points in the tree, you’d divide them into groups like “healthy,” “leukemia,” “lung cancer” and so forth.......Decision trees are used in software that make medical diagnoses by narrowing down someone’s symptoms.
Walmart’s machine learning algorithm found that, “If you buy diapers, you are more likely to buy beer.”
You can prevent effective algorithms from overfitting by keeping models open and restricting assumptions.
Bayesianism is another popular branch of machine learning, and its followers are practically religious in their devotion.....Contrary to the rationalists, Bayesians are empiricists who believe that logical reasoning is flawed and that true intelligence comes from observation and experimentation.....Their algorithm of choice is called Bayesian inference, which works by keeping a number of different hypotheses or models open simultaneously. The degree to which we believe in any one of these hypotheses or models will vary depending on the evidence found in the data, as some will invariably receive more support than others.
This approach can also help provide a medical diagnosis......The more data the record provides, the more diseases the algorithm can rule out, until one hypothesis becomes the statistical winner.
By ignoring the possible connections between events, Bayesian inference avoids overfitting and becoming too powerful by strictly focusing on the connection between cause and effect.
Similar assumptions are used by voice-recognition software like Siri. When you say, “Call the police!” the Bayesian inference keeps options open and considers how likely it is that you might have said, “Call the please!”
"All models are wrong, but some are useful."
Unsupervised learning algorithms are great at finding structure and meaning in raw data.
Unsupervised learning is a category of algorithms that are designed to use raw and noisy data........Clustering algorithms are one group of unsupervised learners that can discover categories from large amounts of raw data. This is the kind of algorithm that can be used in image recognition or voice isolation software, which can identify a face or object among millions of pixels, or single out a voice in a noisy crowd........Sketch artists, for instance, are able to reproduce faces with such accuracy because they memorize ten different variations of each facial feature–nose, eyes, ears and so on. This narrows their options down considerably, making it possible to produce a passable drawing based on a description alone.
Neural networks are another effective way to crunch massive amounts of raw data.
While other algorithms process data sequentially, neural networks work like a brain and process multiple inputs at the same time.
There is no one perfect algorithm, and a unifying master algorithm is required to tackle the big problems.
For every data set where an algorithm comes up with something useful, a devil’s advocate could use the same algorithm on another data set to show that everything it does is nonsensical. That’s why it’s important to make the right assumptions about the data you’re applying the algorithm to.
The majority of the most difficult problems in computer science are fundamentally related and could be solved with one good algorithm......Some problems that have already been solved include: determining the shortest route to visit several cities, compressing data, controlling urban traffic flow, turning 2D images into 3D shapes, laying out components on a microchip and, last but not least, playing Tetris.
Discovering an efficient solution for one of these problems essentially solved them all.
Unfortunately, the most important problems facing humanity require much more capable algorithms than are currently available.
In modern business, finding the best algorithm and best data is the key to success.
With the internet came virtually unlimited consumer choice, and now the question is: How do you decide what to buy when there are 10 million options?
This is where machine learning comes in and helps narrow things down.
whoever has the best data can come up with the best algorithm, which is why data is a tremendous strategic asset.
The business of buying data has become so big that experts believe data unions and data banks will eventually allow private citizens and companies to conduct fair negotiations about the use of their data......A data union could operate like other worker unions, with like-minded people joining forces to ensure that information is being used fairly and accurately.......This kind of regulation could benefit everyone.
In the future, you’ll have a digital model of yourself to help make life easier.
By sharing all your data with the ultimate master learning algorithm you will end up with a pretty accurate digital model of yourself.....Imagine the master algorithm: Seeded with a database containing all general human knowledge, then personalized with all the data that you’ve collected over the course of your life, including emails, phone records, web searches, purchases, downloads, health records, GPS directions and so on and so on.
In addition to simple things like automated web searches and recommending new books and movies, it could also file your tax returns, pay your credit-card bills, sort your email, plan your vacations and, if you’re single, it could even set you up on dates.
Imagine you are looking for a new job. After spending a second on LinkedIn, your model could apply for every suitable job available, including some perfect jobs you might have otherwise overlooked.
Final summary
The key message in this book: Machine learning algorithms are universal problem solvers that need only a few assumptions and a whole lot of data to work their magic. Unifying the current branches of machine learning into one ultimate master algorithm would advance humanity like no other single event in history. Even as it stands today, advanced algorithms and access to personal data are already crucial for businesses to be competitive.
My take: I found this book to be really interesting. It dealt with the concepts at a fairly high level but with sufficient detail to be able to understand them in general terms. I learned a lot from it. Five stars from me. show less

I'm am a Machine Learning researcher, and this book gives a good introduction in a human way of how can I present some concepts so the people can understand.

But after all that introduction, it's quite boring. I don't believe in some points that the author gave about The Master Algorithm.