Foundations & Combinatorics

Probability: The Language of Uncertainty

Probability is the mathematical framework for quantifying uncertainty. Whether you're a data scientist predicting user churn, a gambler calculating pot odds, or a physicist studying quantum states, you're using the same fundamental rules established centuries ago.

At its heart, probability is about counting: how many ways can something happen versus how many ways it could have happened?

Set Theory & Sample Spaces

Before we can calculate the odds of anything, we need to rigorously define the "universe" of all possible outcomes. This universe is called the Sample Space.

DDefinition 1.1

Sample Space (Ω)

The Sample Space, denoted by $\Omega$ (the Greek letter Omega), is the set of all possible outcomes of an experiment. An Event $E$ is any subset of the sample space ( $E \subseteq \Omega$ ).

Understanding $\Omega$ is critical. If you define your sample space incorrectly, every calculation that follows will be wrong.

EExample

Real-World Sample Spaces

Tossing a Coin: $\Omega = \{H, T\}$ (where $H$ = Heads, $T$ = Tails).
Rolling a Six-Sided Die: $\Omega = \{1, 2, 3, 4, 5, 6\}$ .
Weather Tomorrow: $\Omega = \{\text{Sunny, Rainy, Cloudy, Snowy}\}$ .
Stock Price: $\Omega = [0, \infty)$ . Because a stock price can be any decimal value (like $150.23), we use an interval rather than a list.

Set Operations: The Building Blocks

Since events are just sets of outcomes, we use set theory to combine them:

Union ( $A \cup B$ ): Either $A$ happens, $B$ happens, or both happen. Think of it as the logical "OR".
Intersection ( $A \cap B$ ): Both $A$ and $B$ must happen at the same time. Think of it as the logical "AND".
Complement ( $A^c$ or $A'$ ): The event that $A$ does not happen.

The Three Axioms of Kolmogorov

In 1933, Andrey Kolmogorov established three rules that all probability measures must follow. If these aren't met, the system isn't "probability."

Non-negativity: $P(E) \ge 0$ for any event $E$ . You can't have a -10% chance of rain.
Normalization: $P(\Omega) = 1$ . The probability that something in the universe happens is 100%.
Countable Additivity: If events $A_1, A_2, \ldots$ are mutually exclusive (meaning they can't happen at the same time, so $A_i \cap A_j = \emptyset$ ), then the probability of any of them happening is the sum of their individual probabilities: $P(\bigcup_{i=1}^\infty A_i) = \sum_{i=1}^\infty P(A_i)$

1.By definition, an event

E

and its complement

E^c

are mutually exclusive (

E \cap E^c = \emptyset

2.Together, they cover the entire sample space:

E \cup E^c = \Omega

3.Using Axiom 3 (Additivity):

P(E \cup E^c) = P(E) + P(E^c)

4.Using Axiom 2 (Normalization):

P(E \cup E^c) = P(\Omega) = 1

5.Therefore,

P(E) + P(E^c) = 1

, which gives us the shortcut: $P(E) = 1 - P(E^c)$ . This is often much easier than calculating

P(E)

directly.

∎

Combinatorics: The Art of Counting

How many ways can you arrange a 10-song Spotify playlist? What are the odds of cracking a 4-digit PIN? When the sample space is too large to list manually, we use Combinatorics.

The Fundamental Counting Principle

If there are $n$ ways to do one thing and $m$ ways to do another, there are $n \times m$ ways to do both. This "multiplication of choices" is the engine behind all counting formulas.

\binom{n}{k} = \frac{n!}{k!(n-k)!}

Method	Formula	Order Matters?	Replacement?	Intuition
Permutations	$n^k$	Yes	Yes	Digital PIN Codes or Passwords
Permutations (Strict)	$\frac{n!}{(n-k)!}$	Yes	No	Race Finishers (1st, 2nd, 3rd)
Combinations	$\binom{n}{k}$	No	No	Lottery Numbers / Poker Hands
Stars & Bars	$\binom{n+k-1}{k}$	No	Yes	Distributing identical items (e.g. cookies) to distinct buckets (e.g. kids)

EExample

The Birthday Paradox

In a room of just 23 people, there is a 50.7% chance that at least two people share a birthday. This is counter-intuitive because our brains think about the chance of someone sharing our birthday (which is low), rather than any two people sharing any birthday. There are $\binom{23}{2} = 253$ possible pairs of people in that room, giving many opportunities for a match!

The Password Problem

medium

A hacker knows your 4-digit PIN uses only distinct digits (no repeats) and definitely starts with a '7'. How many possible PINs must they test?

← Previous

Beginning

Course Progression

1 of 25

Conditional Probability & Independence