Markov Chains/"Dynamical Systems"

There are many things in the world that we can model using linear systems of equations. One of these ways to model the world is with something called a Markov Chain (if you take Artifical Intelligence or Natural Language Processing, you'll learn a lot more about these in the future!). A Markov Chain is a model where we have a set of states and a set of transition probabilities.

For example, my world might be composed of two states:

Now, let's imagine that given our measurements about the current state of covid, on any given day, we expect 2% of the people who don't have covid to move to the covid state (they've become infected) and we expect 14% of the people who have covid to move to the no covid state (they've recovered).

We could visualize this like:

image.png

Further, we can fill out this chart to explicitly include information about the probability that a person has that they will stay in a certain state if they were there the day before:

image.png

This is called a markov chain. You'll also hear it referred to in this class as a dynamical system.

Basically, we have a system that changes over time and we know (or have a good guess) about how that system will change given the current state.

Translating a Markov Chain to systems of linear equations

Given the above covid/no covid example, we could model what would happen if we had 100 people total.

If 50 of these people start in "no covid" and 50 start in "covid", at time 0, we have:

image.png

At time 1, we would have:

image-2.png

At time 2, we would have: image-3.png

Since it is clearly reasonable for us to have fractional human beings, this gives us a good way to answer questions like "is covid increasing or decreasing over time?" and "is this a stable system?".

Next, let's write down this system as a set of linear equations, using $c$ to represent covid and $n$ to represent no covid. $c'$ will be the state on the next day for covid; $n'$ will be the state on the next day for no covid:

$$ c' = c - 0.14c + 0.02n \\ n' = n - 0.02n + 0.14c $$

Gathering together like terms:

$$ c' = 0.86c + 0.02n \\ n' = 0.14c + 0.98n $$

Re-writing in vector form:

$$ \begin{bmatrix} c' \\ n' \end{bmatrix} = \begin{bmatrix} 0.86 \\ 0.14 \end{bmatrix} c + \begin{bmatrix} 0.02 \\ 0.98 \end{bmatrix} n $$

Now, let's refer to the state of the system as a whole as $x$ (this is because we want to end up with an equation in the form $Ax = b$, or, specifically $x' = Ax$). We can say that: $$ x' = \begin{bmatrix} c' \\ n' \end{bmatrix} = \begin{bmatrix} 0.86 \\ 0.14 \end{bmatrix} c + \begin{bmatrix} 0.02 \\ 0.98 \end{bmatrix} n $$

And, combining the column vectors: $$ x' = \begin{bmatrix} 0.86 & 0.02\\ 0.14 & 0.98 \end{bmatrix} x $$

Solving for the next day/time step:

(We'll go through the matrix math behind the above example now), starting with 50 people in each state.

$$ x' = \begin{bmatrix} 0.86 & 0.02\\ 0.14 & 0.98 \end{bmatrix} \begin{bmatrix} 50 \\ 50 \end{bmatrix} $$

ICA Question 3: Using matrix math, calculate the number of people at time 3 with covid given a starting state of $ \begin{bmatrix} 50 \\ 10000 \end{bmatrix} $ (fifty people start with covid, 10000 people start with no covid).

Combining with Eigenvalues/vectors

What are the eigen values of our system?

(numpy documentation)

You can solve for these values by hand from the previous part of this lesson on eigenvalues and eigenvectors.

Given these eigenvectors, we know that they fulfill the equation $Av = \lambda I v$., therefore, we can write:

$$ \begin{bmatrix} 0.86 & 0.02\\ 0.14 & 0.98 \end{bmatrix} \begin{bmatrix} -0.70710678 \\ 0.70710678 \end{bmatrix} = 0.84 \begin{bmatrix} 1 & 0\\ 0 & 1 \end{bmatrix} \begin{bmatrix} -0.70710678 \\ 0.70710678 \end{bmatrix} $$

We'll work on another example of this in class on Thursday!