QUASISTATIONARITY IN MARKOVIAN MODELS

by

Andrew Hart, The University of Queensland
Phil Pollett, The University of Queensland
Pauline Schrijner, The University of Durham

Fig.1: The behaviour we seek to describe

MARKOV CHAINS WITH POSITIVE DRIFT

Let be a continuous time Markov chain over a denumerable state space S. For simplicity take .

Let be the matrix of transition rates (q-matrix), assumed to be stable and conservative, so that , for , represents the transition rate from state i to state j and , where represents the transition rate out of state i.

Let , where

and suppose that . It will not be necessary to assume that the transition function P is determined uniquely by Q.

We assume only that P is irreducible, that is, for all , and transient, that is,

LIMITING CONDITIONAL DISTRIBUTIONS

We shall be concerned with the existence of a limiting conditional distribution (LCD) for the process:

The conditional probability can be evaluated as follows:

where

Note that and (by irreducibility and transience) , .

LCDs for ABSORBING CHAINS

Compare the above definition with the widely studied LCDs for absorbing chains:

Append to S an absorbing state (or, more generally, an absorbing set) such that for some (and then all) and t>0.

In this context we study the limit (as ) of

or, more generally (when , the probability of absorption starting in j, is less than 1),

HITTING PROBABILITIES

Let , so that . This can be evaluated as

Claim. , , .

It follows that is a subinvariant vector for P; in particular,

where .

Claim. b is a subinvariant vector for Q; in fact, b is `almost invariant':

CONSTRUCTION OF THE DUAL

Define and by

and

Clearly is a (standard) transition function and is a q-matrix such that . However, P is dishonest with

and is non-conservative with

(Recall that .)

CONSTRUCTION OF THE DUAL

Append to S an absorbing state and extend the definition of and to as follows:

and

In this way will be conservative and will be honest.

is a -transition function since, for ,

INTERPRETATION OF THE DUAL

Let L be the last exit time of state 0, that is,

and define by

is killed at time L. Then, since , is Markovian with transition function , and, if , its initial distribution is given by

Notice that is absorbed with probability 1:

EXISTENCE OF LCDs

By the definition of , we see immediately that, for every ,

Here is one such result for absorbing chains:

Theorem. Suppose S is -positive recurrent and let be the (essentially unique) -invariant measure for . Then, if ,

-CLASSIFICATION

and P have the same -classification.

The decay parameter:

Note that ( ) is the same for each and , .

Geometric ergodicity: .

-transience/recurrence: for ,

-positive/null recurrence: for ,

-INVARIANT MEASURES FOR P

Definition. A collection of positive numbers is called a -subinvariant measure for P if, for all ,

and -invariant if, for all ,

Now, for any given collection , define by .

Claim. m is -(sub)invariant for P if and only if is -(sub)invariant for .

Claim. If m is a -subinvariant measure for P, then if . If m is -invariant for P, then only if .

-INVARIANT MEASURES FOR Q

Definition. A collection of positive numbers is called a -invariant measure for Q if, for all ,

Again we have that m is -invariant for Q if and only if is -invariant for  .

Now suppose that Q is regular, so that P, now being the unique Q-transition function, is honest.

We can then apply to the following result for absorbing chains (with a transient class S and absorbing state ):

Theorem. If is a finite -invariant measure for , then it is -invariant for if and only if

LCDs WHEN Q IS REGULAR

Theorem. Let and suppose that is a -invariant measure for Q. Then, , and m is -invariant for P if (and only if)

whence if S is -positive the LCD exists:

Other examples:

1. Kesten's bounded jump conditions.

2. Asymptotic remoteness.

3. Results for special processes: birth-death processes, branching processes, catastrophe processes.

BIRTH-DEATH PROCESSES

Consider an irreducible birth-death process on . Its q-matrix is given by

where ( ), ( ) and .

Define series A and C by

where and, for ,

Assume that (Q regular) and that (S transient). Then, the hitting probabilities are given by

Define polynomials , where , by , and, for , , and let , , where is the decay parameter of S. Then, m is the essentially unique -invariant measure.

The dual is an absorbing birth-death process; we get (in an obvious notation):

A direct application of van Doorn's result for absorbing birth-death processes shows that the LCD exists if and only if :

AN EXAMPLE

Let , V>0 and , and define ,

where .

This defines a density-dependent birth-death process with constant jump probabilities: think of V as ``area'' and n/V as ``population density''. The idea is that, as V gets large, the density process becomes ``more deterministic'':

Clearly the process is irreducible if f(n)>0 for all , and then transient if (and only if) , in which case

Two choices for f:

Each has and .

Fig.1: Simulation

Fig.2: Limiting conditional distribution

Fig.3: Simulation

Fig.4: Limiting conditional distribution

Fig.5: Simulation

Fig.6: Limiting conditional distribution

Fig.7: Simulation

Fig.8: Limiting conditional distribution