Hi, in this set of lecture we're going to
talk about Markov Models. Now Markov
Models are really simple, they consist of
just two parts. The first thing is there's
a set of states, so those are states that
a person's psyche could be in. They could
be the state of a particular government or
an economy. And then there's going to be
transition probabilities and the
transition probabilities are going to tell
us the probability of moving from one
state to another. So remember all the
stuff we learned about probabilities. How
they sum to one, and that sort of thing.
All those rules are gonna apply here. But
the probabilities are gonna tell us how
likely it is to move from, say, one state
to another. Let me give a couple examples.
So first, let's suppose we have some
students. And those students could be in
either one of two states. They could be
alert, or they could be bored. Now,
there's gonna be some probability, P, that
they move from alert to bored. And maybe
some probability Q that they move from
bored to alert. And over time, students
are gonna be moving back and forth from
the alert state to the bored state. And
the Markov process will give us A
framework with which to understand how
those dynamics take place. Now, alert and bored students 
don't seem maybe that important. So let's do
something more relevant. Let's talk about
countries being free or not free. So
those are the two states. A country can be
free. A country can be not free. Now what
we can do is, we can [inaudible] data [inaudible]
what's the probability that a state moves
from free to not free, and what's the
probability that a state moves from not
free to free? That'll also be, a
Markov process. So if we look at
historically the number of free states and
not free states and then create a
third category called "partly free", which is
the red line, we can see that there's these
different trends, right? The free
states seem to be increasing. The not-free
tend to be decreasing. What we can do is,
we can use Markov process to figure out,
where is this process going to end up. Is
this going to end up with all free states?
Or are we going to end up with maybe some
moderate number of free states in the
process still churning? That's where the
model's going to help us. Now remember we
talked about the different sorts of things
that processes can do? They can go to
equilibria, they can be cycles, they can
be completely random, or they can be
complex. What we're going to find, is that
as long as just a few assumptions hold,
that Markov processes are
gonna be here: they're gonna go to
equilibrium. And so there's a theorem
called the Markov convergence theorem.
This Markov convergence theorem tells
us: as long as a couple really mild
assumptions hold, namely that we have, like a
finite number of states, and those
probabilities stay fixed. And then
one other thing, you can get from any
position, any state, to any other state.
Then what we'll get, is the system goes to
an equilibrium. So this is a really powerful
thing and has all sorts of implications
that we're gonna flesh out as we look 
more deeply into the model. Now to do this, to
understand Markov processes, we're going
to have to introduce a little bit more notation,
another technique, another tool from the
study of models. And these are called
matrices. So matrices are really just like
a little grid, make it two by two, or
three by three. Where you put numbers in
here, like point four, point five, point
six, point five. And those will be the
transition probabilities. So what we're
gonna learn how to do, is we're going to
learn how to multiply by matrices in order
to understand these Markov processes, and
in particular to understand how the Markov
convergence theorem works. We'll use
these matrices to explain why these
systems go to equilibria. Now the reason
we do Markov processes is twofold. One is,
they're really sort of a useful
way to think about how the world works and
we get this really powerful result, the
Markov convergence theorem that says these
systems are going to go to these, this,
these equilibria. Any Markov process goes
to an equilibrium. Second reason we're
going to do them, is what we talked about
in the previous lecture, it's this idea of
exaptation. That the Markov model's
incredibly fertile. Once we have the
Markov idea in our head, once we
understand what a Markov process is, we
can apply it in a whole bunch of different
settings. In fact one of my colleagues, we
give him almost anything, he'll say "that's
a Markov process". And there's a sense in
which a lot of things are Markov processes,
and it's often really, really useful to
think of things in the context of Markov
processes. It's also true, once you have
this idea of transition probabilities and
matrices we see can use those in a lot of
settings as well. Okay, so let's get
started. I'm going to start out just by a
very, very simple Markov process, then
what we're going to do is we're gonna look
at a slightly more complicated one and
then see how the Markov convergence
theorem works. Once we've got all that in
play, then we'll go back and talk about
exaptation, where we can apply other
settings. Okay, thanks.