Conditional Probability & Independence

Updating Your Beliefs: Conditional Probability

How does knowing it's cloudy change the chance of rain? If you know a person likes sci-fi movies, does that change the chance they like video games? Probability is not just about fixed numbers; it's about information.

DDefinition

Conditional Probability

The probability of $A$ given that $B$ has already occurred is: $P(A \mid B) = \frac{P(A \cap B)}{P(B)}$

We are essentially shrinking the "universe" from the full $\Omega$ to just the subset where $B$ happened. $P(B)$ must be greater than zero for this to be defined.

✦Intuition

The Restricted Universe

Think of the sample space as a classroom. If I ask "What is the probability of picking someone taller than 6ft?", my universe is the whole class. If I ask "What is the probability they are taller than 6ft given they play basketball?", I've thrown out everyone who doesn't play basketball. My "universe" is now much smaller (the basketball players), and the probability of being tall is likely much higher within that group.

Multiplication Rule

By rearranging the definition, we get the rule for finding the probability that both $A$ and $B$ happen:

P(A \cap B) = P(B)P(A \mid B) = P(A)P(B \mid A)

Independence: The Absence of Information

Events $A$ and $B$ are Independent if knowing $B$ happened tells you absolutely nothing about $A$ .

!Common pitfall

Common Confusion: Independent vs. Mutually Exclusive

This is the most common trap for students!

Independent: $P(A \mid B) = P(A)$ . One event has no influence on the other. (e.g., Tossing a coin twice).
Mutually Exclusive (Disjoint): $P(A \cap B) = 0$ . If $A$ happens, $B$ cannot happen. They are highly dependent! (e.g., Being in London and New York at the same time). If you know you are in London, the probability you are in New York is 0.

Defining Independence Mathematically

Two events $A$ and $B$ are independent if and only if:

P(A \cap B) = P(A)P(B)

Breaking Down Complexity: Law of Total Probability

Calculating a probability directly is often hard. The Law of Total Probability (LOTP) lets us break the world into distinct "scenarios" (a partition) and solve each one individually.

✦Intuition

The Partition Strategy

If you have a set of mutually exclusive events $B_1, B_2, \ldots, B_n$ that cover the whole sample space (a partition), then for any event $A$ : $P(A) = \sum_{i=1}^n P(A \cap B_i) = \sum_{i=1}^n P(A \mid B_i)P(B_i)$

Email Filter Logic

Using LOTP to find the total probability an email is 'Spam' based on keywords.

Contains keyword "Free" → Is actually Spam

Path probability: 0.18

Contains keyword "Free" → Is actually Not Spam

Path probability: 0.02

Does Not Contain keyword "Free" → Is actually Spam

Path probability: 0.04

Does Not Contain keyword "Free" → Is actually Not Spam

Path probability: 0.76

Bayes' Theorem: The Logic of Science

Bayes' Theorem is the mathematical engine behind medical diagnosis, spam filters, and self-driving cars. It allows us to "reverse" conditional probabilities: going from $P(\text{Effect} \mid \text{Cause})$ to $P(\text{Cause} \mid \text{Effect})$ .

P(\text{Cause} \mid \text{Effect}) = \frac{P(\text{Effect} \mid \text{Cause}) \cdot P(\text{Cause})}{P(\text{Effect})}

✦Intuition

Bayesian Thinking

Prior ( $P(C)$ ): What did we believe before the evidence?
Likelihood ( $P(E \mid C)$ ): How well does the evidence match our theory?
Evidence ( $P(E)$ ): How likely is the evidence in general?
Posterior ( $P(C \mid E)$ ): What should we believe now?

The Rare Disease Paradox

medium

A disease affects 1% of the population. A test is 95% accurate (it catches 95% of diseases and only misidentifies 5% of healthy people). You test positive. Should you panic?

← Previous

Foundations & Combinatorics

Course Progression

2 of 25

Random Variables & Expectations