You see, but you do not observe — Sherlock Holmes
I’m a fan of Sherlock Holmes, the genius detective of London, best know for his deductions, logical reasoning, and observational skills.
What I loved most was how he explained he came to such crazy deductions with little evidence that border on the fantastic when investigating cases. But you know, it’s fantastic not reality, or maybe not?
Well, what if I tell you there is a formula that can be used for treasure hunting, logical deduction, machine learning prediction models, evaluating a possible virus in your body, or just finding your lost phone at home. Let me introduce you the Bayes’ theorem.
An Introduction to Bayes’ theorem: The base of logical deduction
A study conducted by the two psychologists, Daniel Kahneman and Amos Tversky has this popular question in their book Thinking, Fast and Slow:
“Steve is very shy and withdrawn, invariably helpful but with very little interest in people or the world of reality. A meek and tidy soul, he has a need for order and structure and a passion for detail.” Is Steve more likely to be a librarian or a farmer?
Probably your answer is: “a librarian” and most people do. This is because we quickly associate a meek and tidy soul with a librarian-like stereotype. In the study paper, people who answer “a librarian” don’t incorporate information about the ratio of farmers in their judgments. According to Daniel Kahneman and Amos Tversky, the ratio of farmers to librarians in the USA it’s about 20 to 1, which means: There are 20 farmers per librarian.
You probably say: “Who knows that ratio? how can people expect that” and you are not wrong, but this question is whether people consider that ratio to make a deduction. In the experiment, Daniel Kahneman wants to know what’s are our priors to make an irrational approximation.
I am using here the same approach as the 3Blue1Brown channel because I think the best way to teach numbers is with pictures and shapes. So, all the credits for the channel.
So, how do we solve that puzzle?
Start with a representative sample: Let’s say 200 farmers and 10 librarians according to the ratio of 20:1.
Match the probabilities: What percentage of librarians and farmers fit the description “meek and tidy soul”. Let’s say, 50% of librarians fit the description and 10% of farmers fit the description. We know for sure that 100% of people in the sample don’t fit the description for sure.
With this sample, we expect that about 5 librarians and 20 farmers fit the description. Finally, we solve this question using the next formula of conditional probability.
Or if you prefer shapes:
Even if a librarian is 5 times as likely as a farmer to fit the description, it only has a 20% chance, quite low compared to farmers that fit the description. Sounds irrational at first time, but with new evidence, you have completely changed the way you see the problem.
This is the heart of Bayes’ Theorem: For any evidence you hold, you should continually update your conclusions.
- First, you think Steve is a librarian,
- then you see some evidence and you noticed there are more farmers per librarian,
- and finally, you change your belief. Easy peasy.
Bayes’ Theorem in depth
I hope you get the idea of conditional probability with that explanation, now, let me introduce you to the Bayes formula.
A, B: Events
P(A|B): The probability of A given B is true
P(B|A): The probability of B given A is true
P(A), P(B): The independent probabilities of A and B
Don’t be scared, it looks like a different thing from what I just explained, but in essence, it is the same.
Event (A): Being a Librarian
Event (B): Fit the description
P(A): Probability of being a Librarian
P(B|A): Probability of being a Librarian fitting the description
P(B|A)P(A) + P(B| -A)P( -A): Probability of being a Librarian fitting the description + Probability of fitting the description but not being a Librarian.
So, the probability of a librarian given the description is equal to:
Do you see? Easy peasy.
Bayes Theorem in real problems
Probably you don’t see the importance of that simple formula, but with it, Tommy Thompson was able to find the rest of the SS Central America ship at the bottom of the ocean, which had tons of gold in it.
The way they found it was ingenious: The 1,400 square miles of ocean were divided into a two-mile-wide grid, and a map was created showing the probability of each square finding a ship at a particular location. The map indicated where to look first, and when a clue was found, Bayes’ theorem was used to update all probabilities of finding the treasure in each square as they moved from one location to another.
This simple formula helps Thompson’s team draw inferences from uncertain or incomplete evidence.
Bayes’ theorem in Machine Learning Models
Most current inference systems in machine learning use Bayes’ theorem because of its use in probability frameworks for fitting a model to a training data set. Is a very popular classification algorithm for real-time prediction, recommendation systems, text classification models, and so on.
Making better company decisions
In 1996 Bill Gates famously declared:
“Microsoft’s competitive advantage is its expertise in Bayesian networks.” (Los Angeles Times, October 28, 1996)
Bayesian philosophy in approaching various scientific and practical problems has become a convincing approach within the modern framework of reasoning and decision-making processes. Huge companies like Google, Amazon, Netflix, etc. Use the same philosophy for developing new products and managing risk.
Final thoughts
Bayes’ theorem is everywhere, from statistics and artificial intelligence to the hunt for a treasure hidden in the depths of the sea, it is the base of all logical deduction, is the base of all scientific research, and it is so simple that it can be applied to our daily lives, applying this philosophy update our beliefs, our way of thinking and acting. I hope this article has been helpful and changed your mindset to take nothing for granted.
Having good judgments, making accurate predictions, and making good decisions, it’s mostly about which mindset you’re in — Julia Galef
See you next time 😉
Originally published at https://juandiegorr.com on June 2, 2022.
