16.1 Elements of markov sequences  (Page 9/13)

The following important conditions are intuitive and may be established rigorously:

• $i\leftrightarrow j$ implies i is recurrent iff j is recurrent
• $i\to j$ and i recurrent implies $i\leftrightarrow j$
• $i\to j$ and i recurrent implies j recurrent

Limit theorems for finite state space sequences

The following propositions may be established for Markov sequences with finite state space:

• There are no null states, and not all states are transient.
• If a class of states is irreducible (i.e.,has no proper closed subsets), then
  • All states are recurrent
  • All states are aperiodic or all are periodic with the same period.
  • If a class C is closed, irreducible, and i is a transient state (necessarily not in C ),
    • then $F(i,j)=F(i,k)$ for all $j,k\in C$ .

A limit theorem

If the states in a Markov chain are ergodic (i.e., positive, recurrent, aperiodic), then

If, as above, we let

the result above may be written

where

Each row of ${\displaystyle {\mathbf{P}}_0=\underset{n}{lim}{\mathbf{P}}^n}$ is the long run distribution ${\displaystyle \pi =\underset{n}{lim}\pi \left(n\right)}$ .

Definition . A distribution is stationary iff

The result above may be stated by saying that the long-run distribution is the stationary distribution. A generating function analysisshows the convergence is exponential in the following sense

where $\left|\lambda \right|$ is the largest absolute value of the eigenvalues for P other than $\lambda =1$ .

The long run distribution for the inventory example

We use MATLAB to check the eigenvalues for the transition probability P and to obtain increasing powers of P . The convergence process is readily evident.