Bayes Theorem

Definition

Bayes’ rule or Bayes’ theorem is the law of probability governing the strength of evidence – the rule saying how much to revise our probabilities (change our minds) when we learn a new fact or observe new evidence. ¹

Bayes’ theorem, named after 18th-century British mathematician Thomas Bayes, is a mathematical formula for determining conditional probability. Conditional probability is the likelihood of an outcome occurring, based on a previous outcome occurring. Bayes’ theorem provides a way to revise existing predictions or theories (update probabilities) given new or additional evidence. In finance, Bayes’ theorem can be used to rate the risk of lending money to potential borrowers.

Applications of the theorem are widespread and not limited to the financial realm. As an example, Bayes’ theorem can be used to determine the accuracy of medical test results by taking into consideration how likely any given person is to have a disease and the general accuracy of the test. Bayes’ theorem relies on incorporating prior probability distributions in order to generate posterior probabilities. Prior probability, in Bayesian statistical inference, is the probability of an event before new data is collected. This is the best rational assessment of the probability of an outcome based on the current knowledge before an experiment is performed. Posterior probability is the revised probability of an event occurring after taking into consideration new information. Posterior probability is calculated by updating the prior probability by using Bayes’ theorem. In statistical terms, the posterior probability is the probability of event A occurring given that event B has occurred.

Bayes’ theorem thus gives the probability of an event based on new information that is, or may be related, to that event. The formula can also be used to see how the probability of an event occurring is affected by hypothetical new information, supposing the new information will turn out to be true. For instance, say a single card is drawn from a complete deck of 52 cards. The probability that the card is a king is four divided by 52, which equals 1/13 or approximately 7.69%. Remember that there are four kings in the deck. Now, suppose it is revealed that the selected card is a face card. The probability the selected card is a king, given it is a face card, is four divided by 12, or approximately 33.3%, as there are 12 face cards in a deck. ²

Who

In all the statistics courses I took as an undergraduate and a graduate none of them included a discussion of Bayes’ theorem. The problem is that an understanding and application would have made my life easier. I had to learn it on my own. This is for all of you where Bayes’ theorem is not taught in your statistics classes.

What

“An Intuitive (And Short) Explanation Of Bayes’ Theorem – BetterExplained”. 2021. betterexplained.com. https://betterexplained.com/articles/an-intuitive-and-short-explanation-of-bayes-theorem/.

Today we’re going to talk about Bayes Theorem and Bayesian hypothesis testing. Bayesian methods like these are different from how we’ve been approaching statistics so far, because they allow us to update our beliefs as we gather new information – which is how we tend to think naturally about the world. And this can be a really powerful tool, since it allows us to incorporate both scientifically rigorous data AND our previous biases into our evolving opinions.

How Bayesian Inference Works

Part of the End-to-End Machine Learning School Course 191, Selected Models and Methods at https://e2eml.school/191

Why

See Theoretical Knowledge Vs Practical Application.

How

Many of the References, Additional Reading, websites and YouTube videos will assist you with understanding and applying Bayes’ Theorem.

As some professors say: “It is intuitively obvious to even the most casual observer.”

References

¹ “Bayes’ Rule”. 2022. arbital.com. https://arbital.com/p/bayes_rule/?l=1zq.

² “Bayes’ Theorem”. 2021. Investopedia. https://www.investopedia.com/terms/b/bayes-theorem.asp.

Application

“Bayes’ Rule Applied”. 2018. Medium. https://towardsdatascience.com/bayes-rule-applied-75965e4482ff.

“Bayes’ Rule With A Simple And Practical Example”. 2020. Medium. https://towardsdatascience.com/bayes-rule-with-a-simple-and-practical-example-2bce3d0f4ad0.

“Bayes’ Theorem”. 2021. faculty.washington.edu. http://faculty.washington.edu/tamre/BayesTheorem.pdf.

“Bayes’ Theorem”. 2021. Corporate Finance Institute. https://corporatefinanceinstitute.com/resources/knowledge/other/bayes-theorem/.

“Bayes Theorem (Easily Explained W/ 7 Examples!)”. 2021. calcworkshop. https://calcworkshop.com/probability/bayes-theorem/.

“Bayes Theorem Explained With Example – Complete Guide | Upgrad Blog”. 2021. Upgrad Blog. https://www.upgrad.com/blog/bayes-theorem-explained-with-example-complete-guide/.

“Bayes Theorem – Statement, Proof, Formula, Derivation & Examples”. 2021. BYJUS. https://byjus.com/maths/bayes-theorem/.

Oleszak, Michael. “Bayesian Tricks For Everyday Use”. 2022. Medium. https://towardsdatascience.com/bayesian-tricks-for-everyday-use-2b51da45f3ec.

Sometimes we wish we knew something that we don’t. Unfortunately, in many cases, there is no time or even no way to learn what we need. Nevertheless, decisions and assessments need to be made with only disposable knowledge. While to many of us navigating in the mist of incomplete information seems scary, those who understand the nature of uncertainty can use it to their own advantage. Read on to see how to do it with the help of a few tricks that squeeze the most knowledge from limited information.

“What Is Bayes Theorem | Applications Of Bayes Theorem”. 2019. Analytics Vidhya. https://www.analyticsvidhya.com/blog/2019/06/introduction-powerful-bayes-theorem-data-science/.

Additional Reading

“Bayes’ Theorem”. 2021. mathsisfun.com. https://www.mathsisfun.com/data/bayes-theorem.html.

“Bayes’ Theorem And Conditional Probability | Brilliant Math & Science Wiki”. 2021. brilliant.org. https://brilliant.org/wiki/bayes-theorem/.

“Bayes’ Theorem In Plain English”. 2021. Medium. https://benjaminobi.medium.com/bayes-theorem-in-plain-english-eeb142710475.

“Bayes’ Theorem, Clearly Explained With Visualization”. 2021. Medium. https://towardsdatascience.com/bayes-theorem-clearly-explained-with-visualization-5083ea5e9b14.

“Bayes Theorem Is Easy To Prove, Hard To Understand | Long Nguyen”. 2021. Long Nguyen. https://www.longuyen.de/posts/006-conjugate-prior/.

“Deriving Bayes’ Theorem The Easy Way”. 2022. Medium. https://towardsdatascience.com/deriving-bayes-theorem-the-easy-way-59f0c73496db.

“The Bayes Formula Explained In Pure English”. 2021. Medium. https://towardsdatascience.com/the-bayes-formula-explained-in-pure-english-151c9401a5f8.

“Understanding Bayes Theorem With Ratios – BetterExplained”. 2021. betterexplained.com. https://betterexplained.com/articles/understanding-bayes-theorem-with-ratios/.

Videos

Bayes Theorem in Probability with Examples | Easiest Trick to Understand Bayes Rule in Probability

Bayes theorem probability theory, bayes law probability, bayes rule in probability. In this video, learn the easiest tricks to understand bayes law used in probability with solved example and detailed explanation. This video will clear all your concepts about bayes law.