QUASISTATIONARITY IN MARKOVIAN MODELS
by
Andrew Hart, The University of Queensland
Phil Pollett, The University of Queensland
Pauline Schrijner, The University of Durham
Fig.1: The behaviour we seek to describe
MARKOV CHAINS WITH POSITIVE DRIFT
Let be a continuous time Markov chain over a denumerable state space S. For simplicity take .
Let be the matrix of transition rates (q-matrix), assumed to be stable and conservative, so that , for , represents the transition rate from state i to state j and , where represents the transition rate out of state i.
Let , where
and suppose that . It will not be necessary to assume that the transition function P is determined uniquely by Q.
We assume only that P is irreducible, that is, for all , and transient, that is,
LIMITING CONDITIONAL DISTRIBUTIONS
We shall be concerned with the existence of a limiting conditional distribution (LCD) for the process:
The conditional probability can be evaluated as follows:
where
Note that and (by irreducibility and transience) , .
LCDs for ABSORBING CHAINS
Compare the above definition with the widely studied LCDs for absorbing chains:
Append to S an absorbing state (or, more generally, an absorbing set) such that for some (and then all) and t>0.
In this context we study the limit (as ) of
or, more generally (when , the probability of absorption starting in j, is less than 1),
HITTING PROBABILITIES
Let , so that . This can be evaluated as
Claim. , , .
It follows that is a subinvariant vector for P; in particular,
where .
Claim. b is a subinvariant vector for Q; in fact, b is `almost invariant':
CONSTRUCTION OF THE DUAL
Define and by
and
Clearly is a (standard) transition function and is a q-matrix such that . However, P is dishonest with
and is non-conservative with
(Recall that .)
CONSTRUCTION OF THE DUAL
Append to S an absorbing state and extend the definition of and to as follows:
and
In this way will be conservative and will be honest.
is a -transition function since, for ,
INTERPRETATION OF THE DUAL
Let L be the last exit time of state 0, that is,
and define by
is killed at time L. Then, since , is Markovian with transition function , and, if , its initial distribution is given by
Notice that is absorbed with probability 1:
EXISTENCE OF LCDs
By the definition of , we see immediately that, for every ,
Here is one such result for absorbing chains:
Theorem. Suppose S is -positive recurrent and let be the (essentially unique) -invariant measure for . Then, if ,
-CLASSIFICATION
and P have the same -classification.
The decay parameter:
Note that ( ) is the same for each and , .
Geometric ergodicity: .
-transience/recurrence: for ,
-positive/null recurrence: for ,
-INVARIANT MEASURES FOR P
Definition. A collection of positive numbers is called a -subinvariant measure for P if, for all ,
and -invariant if, for all ,
Now, for any given collection , define by .
Claim. m is -(sub)invariant for P if and only if is -(sub)invariant for .
Claim. If m is a -subinvariant measure for P, then if . If m is -invariant for P, then only if .
-INVARIANT MEASURES FOR Q
Definition. A collection of positive numbers is called a -invariant measure for Q if, for all ,
Again we have that m is -invariant for Q if and only if is -invariant for .
Now suppose that Q is regular, so that P, now being the unique Q-transition function, is honest.
We can then apply to the following result for absorbing chains (with a transient class S and absorbing state ):
Theorem. If is a finite -invariant measure for , then it is -invariant for if and only if
LCDs WHEN Q IS REGULAR
Theorem. Let and suppose that is a -invariant measure for Q. Then, , and m is -invariant for P if (and only if)
whence if S is -positive the LCD exists:
Other examples:
BIRTH-DEATH PROCESSES
Consider an irreducible birth-death process on . Its q-matrix is given by
where ( ), ( ) and .
Define series A and C by
where and, for ,
Assume that (Q regular) and that (S transient). Then, the hitting probabilities are given by
Define polynomials , where , by , and, for , , and let , , where is the decay parameter of S. Then, m is the essentially unique -invariant measure.
The dual is an absorbing birth-death process; we get (in an obvious notation):
A direct application of van Doorn's result for absorbing birth-death processes shows that the LCD exists if and only if :
AN EXAMPLE
Let , V>0 and , and define ,
where .
This defines a density-dependent birth-death process with constant jump probabilities: think of V as ``area'' and n/V as ``population density''. The idea is that, as V gets large, the density process becomes ``more deterministic'':
Clearly the process is irreducible if f(n)>0 for all , and then transient if (and only if) , in which case
Two choices for f:
Each has and .
Fig.1: Simulation
Fig.2: Limiting conditional distribution
Fig.3: Simulation
Fig.4: Limiting conditional distribution
Fig.5: Simulation
Fig.6: Limiting conditional distribution
Fig.7: Simulation
Fig.8: Limiting conditional distribution