QUASISTATIONARITY IN MARKOVIAN MODELS
by
Andrew Hart, The University of Queensland
Phil Pollett, The University of Queensland
Pauline Schrijner, The University of Durham
Fig.1: The behaviour we seek to describe
MARKOV CHAINS WITH POSITIVE DRIFT
Let be a continuous time Markov chain over a denumerable
state space S. For simplicity take
.
Let be the matrix of transition rates
(q-matrix), assumed to be stable and conservative,
so that
, for
, represents the
transition rate from state i to state j and
, where
represents the transition rate
out of state i.
Let , where
and suppose that .
It will not be necessary to assume that the transition function P is
determined uniquely by Q.
We assume only that P is irreducible, that is,
for all
,
and transient, that is,
LIMITING CONDITIONAL DISTRIBUTIONS
We shall be concerned with the existence of a limiting conditional distribution (LCD) for the process:
The conditional probability can be evaluated as follows:
where
Note that and (by
irreducibility and transience)
,
.
LCDs for ABSORBING CHAINS
Compare the above definition with the widely studied LCDs for absorbing chains:
Append to S an absorbing state (or, more generally, an absorbing
set) such that
for some (and then all)
and t>0.
In this context we study the limit (as ) of
or, more generally (when , the probability of
absorption starting in j, is less than 1),
HITTING PROBABILITIES
Let , so
that
. This can be evaluated as
Claim.
,
,
.
It follows that is a subinvariant vector for P;
in particular,
where .
Claim. b is a subinvariant vector for Q; in fact, b is `almost invariant':
CONSTRUCTION OF THE DUAL
Define
and
by
and
Clearly is a (standard) transition function
and
is a q-matrix such that
.
However, P is dishonest with
and is non-conservative with
(Recall that .)
CONSTRUCTION OF THE DUAL
Append to S an absorbing state
and extend the definition
of
and
to
as follows:
and
In this way will be conservative and
will be honest.
is a
-transition function since, for
,
INTERPRETATION OF THE DUAL
Let L be the last exit time of state 0, that is,
and define by
is
killed at time L.
Then
,
since
,
is Markovian with transition function
, and,
if
, its initial distribution is given by
Notice that is absorbed with probability 1:
EXISTENCE OF LCDs
By the definition of , we see immediately that,
for every
,
Here is one such result for absorbing chains:
Theorem. Suppose S is -positive
recurrent and let
be the (essentially unique)
-invariant measure for
. Then, if
,
-CLASSIFICATION
and P have the same
-classification.
The decay parameter:
Note that (
) is the same for each
and
,
.
Geometric ergodicity: .
-transience/recurrence:
for
,
-positive/null recurrence:
for
,
-INVARIANT MEASURES FOR P
Definition. A collection of positive numbers
is called a
-subinvariant measure
for P if, for all
,
and -invariant if, for all
,
Now, for any given collection , define
by
.
Claim. m is -(sub)invariant for P if and only if
is
-(sub)invariant for
.
Claim.
If m is a -subinvariant measure for P, then
if
.
If m is
-invariant for P, then
only if
.
-INVARIANT MEASURES FOR Q
Definition. A collection of positive numbers
is called a
-invariant measure
for Q if, for all
,
Again we have that m is -invariant for Q if and only if
is
-invariant for
.
Now suppose that Q is regular, so that P, now being the unique Q-transition function, is honest.
We can then apply to the following result for absorbing chains
(with a transient class S and absorbing state
):
Theorem.
If is a finite
-invariant
measure for
, then it is
-invariant for
if and only if
LCDs WHEN Q IS REGULAR
Theorem.
Let and suppose that
is
a
-invariant measure for Q. Then,
, and
m is
-invariant for P if (and only if)
whence if S is -positive the LCD exists:
Other examples:
BIRTH-DEATH PROCESSES
Consider an irreducible
birth-death process on . Its q-matrix is given by
where (
),
(
) and
.
Define series A and C by
where and, for
,
Assume that (Q regular) and
that
(S transient). Then, the hitting probabilities
are given by
Define polynomials , where
,
by
,
and, for
,
,
and let
,
, where
is the
decay parameter of S. Then, m is the essentially unique
-invariant measure.
The dual is an absorbing birth-death process; we get (in an obvious notation):
A direct application of van Doorn's
result for absorbing birth-death processes shows that
the LCD exists if and only if
:
AN EXAMPLE
Let ,
V>0 and
, and define
,
where .
This defines a density-dependent birth-death process
with constant jump probabilities: think of V as
``area'' and n/V as ``population density''.
The idea is that, as V gets large, the density process
becomes ``more deterministic'':
Clearly the process is irreducible if f(n)>0 for
all , and then transient if
(and only if)
, in which case
Two choices for f:
Each has and
.
Fig.1: Simulation
Fig.2: Limiting conditional distribution
Fig.3: Simulation
Fig.4: Limiting conditional distribution
Fig.5: Simulation
Fig.6: Limiting conditional distribution
Fig.7: Simulation
Fig.8: Limiting conditional distribution