Until now, we've looked at single snapshots of randomness (like a single coin flip). But most systems in the real world evolve over time. A Stochastic Process is just a sequence of random variables—one for every moment in time: .
- Discrete Time: The closing price of a stock every day ().
- Continuous Time: The fluctuating voltage in a power grid at every millisecond.
- Discrete State: The current floor a lift is on (Floor 1, 2, 3...).
- Continuous State: The exact coordinates of a GPS tracker.
A Markov Chain is the simplest type of time-dependent process. it follows the Markov Property: the future depends only on the state you are in right now, not on the path you took to get there.
Imagine a frog jumping between lily pads. If the frog is on 'Pad A', the probability of its next jump depends only on Pad A. It doesn't matter if it jumped from B or C to get there. The frog has no memory of its journey!
The Transition Matrix
We describe these jumps using a Transition Matrix (), where the entry is the probability of moving from state to state .
| From To | Sunny | Rainy | Cloudy |
|---|---|---|---|
| Sunny | 0.7 | 0.2 | 0.1 |
| Rainy | 0.3 | 0.4 | 0.3 |
| Cloudy | 0.4 | 0.3 | 0.3 |
Markov Chain: Weather Model
If you let a Markov chain run for a very long time, it often settles into a "steady state." This is the Stationary Distribution (). It represents the percentage of time you'll spend in each state in the long run.
The original PageRank algorithm was just a massive Markov chain! It simulated a "random surfer" clicking links on the web. The websites with the highest Stationary Distribution (the pads where the surfer spent the most time) were ranked the highest in search results.
A Poisson Process models events that happen independently and continuously over time at a constant average rate .
- The number of calls arriving at an emergency dispatch center (/hour).
- The number of particles emitted by a radioactive source.
- The number of photons hitting a digital camera sensor.
Poisson Process: Cumulative Arrivals
As time (t) goes on, the total number of events N(t) climbs in discrete steps. A higher rate (λ) means the 'staircase' climbs much more steeply.
A Random Walk is what happens when you take a random step at every tick of the clock. It is the foundation of Financial Mathematics and particle physics.
Imagine starting at 0 and flipping a coin. If Heads, move . If Tails, move .
- Expected Position (): On average, you stay exactly at 0.
- Expected Distance: Even though you average at 0, you drift further away over time. Your average distance from the start grows at a rate of (the square root of the number of steps).
Random Walk Realizations
Three different 'Gamblers' starting with $0. Because every step is random, their paths diverge wildly over time, even though they all have the same 'fair' odds.
In finance, many argue that stock prices follow a random walk. If today's price change is truly random, you cannot predict tomorrow's price based on today's. This is why "beating the market" through timing is so difficult!