16.1 Elements of markov sequences  (Page 11/13)

Arrival times and recurrence times

The basic simulation produces one or more trajectories of a specified length. Sometimes we are interested in continuing until first arrival at (or visit to)a specific target state or any one of a set of target states. The time (in transitions) to reach a target state is one less than the number of stages in the trajectory whichbegins with the initial state and ends with the target state reached.

• If the initial state is not in the target set, we speak of the arrival time .
• If the initial state is in the target set, the arrival time would be zero. In this case, we do not stop at zero but continue until the next visit to a target state (possiblythe same as the initial state). We call the number of transitions in this case the recurrence time .
• In some instances, it may be desirable to know the time to complete visits to a prescribed number of the target states. Again there is a choice of treatment in thecase the initial set is in the target set.

Data files

For use of MATLAB in simulation, we find it convenient to organize the appropriate data in an m-file.

• In every case, we need the transition matrix P . Its size indicates the number of states (say by the length of any row or column).
• If the states are to have values other than the state numbers, these may be included in the data file, although they may be added later, in response to a prompt.
• If long trajectories are to be produced, it may be desirable to determinethe fraction of times each state is realized. A comparison with the long-run probabilities for the chain may be of interest. In this case, the data file maycontain the long-run probability distribution. Usually, this is obtained by taking one row of a sufficiently large power of the transition matrix. This operationmay be performed after the data file is called for but before the simulation procedure begins.

An example data file used to illustrate the various procedures is shown below. These data were generated artificially and have no obvious interpretations in terms of aspecific systems to be modeled. However, they are sufficiently complex to provide nontrivial illustrations of the simulation procedures.