16.1 Elements of markov sequences  (Page 5/13)

Note that $p(0,0)=1$ . If the population ever reaches zero, it is extinct and no more births can occur. Also, if the maximum population (10 in this case) is reached,there is a high probability of returning to that value and very small probability of becoming extinct (reaching zero state).

Inventory problem (continued)

In this case,

Numerical example

To simplify writing, use D for D _n . Because of the invariance with n , set

The various cases yield

$g(0,D)=max\{3-D,0\}$

• $g(0,D)=0$ iff $D\ge 3$ implies $p(0,0)=P(D\ge 3)$
• $g(0,D)=1$ iff $D=2$ implies $p(0,1)=P(D=2)$
• $g(0,D)=2$ iff $D=1$ implies $p(0,2)=P(D=1)$
• $g(0,D)=3$ iff $D=0$ implies $p(0,3)=P(D=0)$

$g(1,D)=max\{1-D,0\}$

• $g(1,D)=0$ iff $D\ge 1$ implies $p(1,0)=P(D\ge 1)$
• $g(1,D)=1$ iff $D=0$ implies $p(1,1)=P(D=0)$
• $g(1,D)=2,3$ is impossible

$g(2,D)=max\{2-D,0\}$

• $g(2,D)=0$ iff $D\ge 2$ implies $p(2,0)=P(D\ge 2)$
• $g(2,D)=1$ iff $D=1$ implies $p(2,1)=P(D=1)$
• $g(2,D)=2$ iff $D=0$ implies $p(2,2)=P(D=0)$
• $g(2,D)=3$ is impossible

$g(3,D)=max\{3-D,0\}=g(0,D)$ so that $p(3,k)=p(0,k)$

The various probabilities for D may be obtained from a table (or may be calculated easily with cpoisson) to give the transition probability matrix

The calculations are carried out “by hand” in this case, to exhibit the nature of the calculations. This is a standard problem in inventory theory, involving costs andrewards. An m-procedure inventory1 has been written to implement the function g .

We consider the case $M=5$ , the reorder point $m=3$ , and demand is Poisson (3). We approximate the Poisson distribution with values up to 20.