Bayes Theorem

Definition

Bayes Theorem is a fundamental principle in probability theory and statistics that describes the probability of an event based on prior knowledge of conditions that might be related to the event. It provides a mathematical formula for updating our beliefs or probabilities as new evidence becomes available.

Main Content

1. The Concept of Conditional Probability

• Conditional probability is the likelihood of an event occurring given that another event has already occurred.
• It is denoted as P(A|B), meaning "the probability of event A occurring given that event B has happened."

2. Prior vs. Posterior Probability

• Prior Probability: This is the initial probability of an event based on current knowledge or historical data before seeing new evidence.
• Posterior Probability: This is the updated probability of the event after incorporating the new evidence using Bayes Theorem.

3. The Bayes Formula Structure

• The formula relates the conditional probabilities of two events.
• It calculates how the "likelihood" of evidence changes the "prior" belief.

       P(A|B) * P(B)
P(B|A) = —————————————
            P(A)

(This diagram represents the mathematical relationship where P(B|A) is the posterior, P(A|B) is the likelihood, P(B) is the prior, and P(A) is the marginal likelihood.)

Working / Process

1. Identify the Events and Data

• Define Event A (the new evidence) and Event B (the hypothesis you are testing).
• Collect the initial values: The prior probability P(B), the likelihood P(A|B), and the total probability of the evidence P(A).

2. Calculate the Marginal Likelihood

• To find P(A), you often sum the probabilities of all ways event A can occur.
• Formula: P(A) = P(A|B) * P(B) + P(A|not B) * P(not B).

3. Apply the Bayes Formula

• Substitute your identified values into the main equation.
• Divide the product of the likelihood and prior by the marginal likelihood to reach the final posterior probability.

Advantages / Applications

• Medical Diagnosis: Doctors use it to calculate the probability of a disease given a positive test result, accounting for the test's accuracy and the disease's prevalence.
• Machine Learning: It powers Naive Bayes classifiers, which are widely used for spam filtering and sentiment analysis.
• Risk Assessment: Used in finance and insurance to update the risk profile of clients as new behavioral data or market changes emerge.

Summary

Bayes Theorem is a mathematical framework used to update the probability of a hypothesis as new evidence is introduced. It bridges the gap between initial assumptions and observed data, allowing for dynamic decision-making under uncertainty.

Important terms to remember:

• Prior: The initial belief.
• Likelihood: How well the evidence supports the hypothesis.
• Posterior: The revised belief after evidence.
• Marginal Likelihood: The total probability of observing the evidence.