
Markov Chains Clearly Explained! Part - 1
What this covers
This 9-minute video from Normalized Nerd is an introduction to Markov chains built around a single running example: a restaurant that serves one food per day, either hamburger, pizza, or hot dog, with each day's choice depending on the previous day's. The chain is drawn as a diagram of weighted arrows, and the video makes the point that the outgoing weights from any state must sum to 1 because they are probabilities. Examples include a 20% chance of hamburgers again (a self-pointing arrow), a 60% chance of pizza after hamburger, and a 0.7 chance of hot dogs after pizza, whatever was served earlier. That last case illustrates the Markov Property, which the video presents as the test for whether a problem can be modeled as a Markov chain.
The second half turns to long-run behavior. The diagram is encoded as a transition matrix, and multiplying a row vector of state probabilities by it gives the next day's probability distribution. The video first finds the stationary distribution by brute force: a random walk starting on a hamburger day gives a rough 4/10, 2/10, 4/10 split after ten steps, and a 100,000-step simulation settles near hamburger 35%, pizza 21%, hot dog 46%. It then calls simulation inefficient and offers a linear algebra route. The stationary condition π·A = π is recognized as an eigenvector equation with eigenvalue 1, solved together with the requirement that the entries sum to 1. The analytic answer closely matches the simulation.
The video also notes that multiple stationary distributions would show up as multiple eigenvectors with eigenvalue 1. It says that special classes of chains with distinctive properties are left for later videos, and it mentions applications in statistics, biology, economics, physics, and machine learning only in passing.
Markov chains are a foundational mathematical concept where future states depend only on the current state, not prior history, making them powerful tools for modeling complex systems across statistics, biology, economics, physics, and machine learning.
- The Markov property (memorylessness) simplifies probabilistic prediction by eliminating dependence on complete historical sequences
- Stationary distributions can be computed efficiently using linear algebra and eigenvector methods rather than expensive simulation
- The framework applies universally across domains because it abstracts the transition logic, not the domain content
This asset isn't compiled yet
You're seeing its claims, ranked. Compile it to build the argument threads, weight them, and check each claim against your library — the full view.
The Markov property states that the probability of reaching state n+1 depends only on state n, not on the complete sequence of steps that came before n, which simplifies tackling complicated real-world problems
“the most important property of the markov chain is that the future state depends only on the current state not the steps before mathematically we can say the probability that n plus 1 x step will be x depends only on the nth step not the complete sequence of steps that came before n at first glance this might not seem so impressive but on a closer look we can see that this property makes our life a lot easier while we tackle some complicated real world problems”
The simulation method for finding stationary distributions is inefficient, and a better approach exists using linear algebra
“but this method doesn't seem to be very efficient is it moreover we don't know if there exists any other stationary states well a better way to approach this problem is linear algebra”
The stationary distribution π is a special row vector that satisfies the equation π·A = π, meaning the probability distribution does not change after multiplying by the transition matrix
“now if there exists a stationary state then after a certain point the output row vector should be exactly same as the input vector let's denote this special row vector as pi so we should be able to write pi a equals pi”
A diagram showing all possible transitions between states, with weighted arrows indicating transition probabilities, constitutes a Markov chain
“now i'm gonna show you all the possible transitions the diagram you are seeing right now is actually a markov chain yes the concept is that simple”
When simulating the restaurant Markov chain for 100,000 steps, the probability distribution converges to fixed values known as the stationary distribution or equilibrium state, meaning this distribution does not change with time
“i wrote a python script to simulate the random work for hundred thousand steps i found that the probabilities converge to these values and this probability distribution has a special name which is the stationary distribution or the equilibrium state this simply means that this probability distribution doesn't change with time for this markov chain”
The equation π·A = π is similar to the eigenvector equation λv = Av (or equivalently Av = λv) with λ = 1, and can be rearranged to show that π is a left eigenvector of A with eigenvalue 1
“if you have ever taken a course of linear algebra then you should find this equation somewhat similar to a very famous equation yes i'm talking about the eigenvector equation consider lambda equal to 1 and just reverse the order of multiplication and you have got our equilibrium state equation how to interpret this you might ask well we can imagine that pi is a left eigenvector of matrix a with eigenvalue equal to 1.”
To determine if a Markov chain has multiple stationary distributions, one needs to check if there exist multiple eigenvectors with eigenvalue equal to 1
“using this technique we can actually see if there are more than one stationary states can you guess how we just need to see if there exists more than one eigenvectors with eigenvalue equal to 1.”
When a row vector π representing a probability distribution is multiplied by the transition matrix A, the result is the probability distribution in the next time step, and this process can be repeated iteratively
“basically pi denotes the probability distribution of the states suppose in the beginning we are on a pizza day so the second element which corresponds to the pizza becomes one and other two remain zero now see what happens when we multiply this row vector with our transition matrix we get the second row of the matrix in other words we get the future probabilities corresponding to the pisa state then we take this result and put it in the place of pi zero we will do this once again”
To find the stationary distribution of a Markov chain, one must solve two equations: the eigenvector equation π·A = π (with eigenvalue 1) and the constraint that the elements of π sum to 1 (because it is a probability distribution)
“the eigenvector must satisfy another condition the elements of pi must add up to 1 because it denotes a probability distribution so after solving these two equations we get the stationary state”
If a restaurant serves only one type of food per day and the choice depends on what was served the previous day, then the next day's choice can be predicted given knowledge of today's choice—for example, there is a 60% chance tomorrow will be a pizza day given today is hamburger
“assume there's a restaurant that serves only three types of foods hamburger pizza and hot dog but they follow a weird rule on any given day they serve only one of these three items and it depends on what they had served yesterday in other words there is a way to predict what they will serve tomorrow for example there is a 60 chance that tomorrow will be a pizza day given that today is a hamburger day”
In the restaurant example, given that pizza was served on day 3, the probability that hot dogs will be served on day 4 is 0.7, regardless of what was served on days 1 and 2
“suppose the restaurant served pizza on the first day hamburger on the second and again pizza on the third day now what is the probability that they will serve hot dogs on the fourth day well we just need to look at the third day and it turns out to be 0.7”
The sum of the weights of all outgoing arrows from any state in a Markov chain must equal 1, because the weights represent probabilities and probabilities must sum to 1
“the second important property is that the sum of the weights of the outgoing arrows from any state is equal to 1. this has to be true because they represent probabilities and for probabilities to make sense they must add up to one”
A Markov chain can be represented as a directed graph, and this graph can be encoded efficiently as a transition matrix (also called an adjacency matrix) where each element represents the weight or transition probability between states
“well this is essentially a directed graph so we can represent it by an adjacency matrix if you don't know what it is then please don't worry it's very simple the elements in the matrix just denote the weight of the edge connecting the two corresponding vertices for example the element in the second row and first column denotes the weight or the transition probability from pizza to hamburger if an element is 0 then it just means that there's no edge between the two vertices this matrix is also called transition matrix”
The analytical linear algebra method for finding the stationary distribution closely matches the values obtained from a 100,000-step simulation, validating both approaches
“now let's compare this result with the previous one that we got from our simulation look how close they are they kind of validate each other”
A row vector π whose elements represent the probabilities of each state is conventionally used to denote the probability distribution of states in a Markov chain
“let me show you first of all let us think about how we can represent this markov chain more efficiently well this is essentially a directed graph so we can represent it by an adjacency matrix if you don't know what it is then please don't worry it's very simple the elements in the matrix just denote the weight of the edge connecting the two corresponding vertices for example the element in the second row and first column denotes the weight or the transition probability from pizza to hamburger if an element is 0 then it just means that there's no edge between the two vertices this matrix is also called transition matrix let's denote it by a remember that our goal is to find the probabilities of each state conventionally we take a row vector called pi whose elements represent the probabilities of the states”
Weighted arrows in a Markov chain diagram represent transitions from one state to another, where the weight indicates the probability of that transition
“i'm gonna represent it in the diagram as weighted arrows the arrow originates from the current state and points to the future state”
There is a 20% chance that tomorrow the restaurant will serve hamburgers again, represented by a self-pointing arrow in the Markov chain diagram
“there's a 20 chance that tomorrow again they will serve hamburgers and that is represented as a self-pointing arrow”
Markov chains are used in many places from statistics to biology, from economics to physics, and in machine learning
“this is a concept that is used in many places from statistics to biology from economics to physics and of course in machine learning”
There are special kinds of Markov chains with unique properties that deserve individual videos but are not covered in this introduction
“i should probably tell you that there are certain markov chains with special properties but they deserve individual videos on their own so here i just gonna stick to the basic kind of markov chains”
For the restaurant Markov chain, the stationary distribution is approximately: hamburger 35%, pizza 21%, hot dog 46%
“it tells us that the restaurant will serve hamburger about 35 of the days pigs are about 21 and hot dog about 46 percent”
Understanding that the Markov property holds (future depends only on present, not history) enables one to identify when a problem can be modeled as a Markov chain
“by now you should have a very good understanding of markov chains and you should be able to identify markov chains successfully”
When performing a random walk on the restaurant Markov chain starting from hamburger day, after 10 steps the probability distribution observed is hamburger 4/10, pizza 2/10, hot dog 4/10
“let's see what they serve for 10 consecutive days so after 10 steps we have a situation like this now we want to find what is the probability corresponding to each of the food items aka the probability distribution of the states well it's pretty simple just divide the number of occurrences of an item by the total number of days for hamburger it's 4 upon 10 for pizza it's 2 upon 10 and it's 4 upon 10 for hot dogs”