QUASISTATIONARITY IN MARKOVIAN MODELS

by

Andrew Hart, The University of Queensland
Phil Pollett, The University of Queensland
Pauline Schrijner, The University of Durham

Fig.1: The behaviour we seek to describe

MARKOV CHAINS WITH POSITIVE DRIFT

Let tex2html_wrap_inline454 be a continuous time Markov chain over a denumerable state space S. For simplicity take tex2html_wrap_inline458 .

Let tex2html_wrap_inline460 be the matrix of transition rates (q-matrix), assumed to be stable and conservative, so that tex2html_wrap_inline464 , for tex2html_wrap_inline466 , represents the transition rate from state i to state j and tex2html_wrap_inline472 , where tex2html_wrap_inline474 represents the transition rate out of state i.

Let tex2html_wrap_inline478 , where

displaymath480

and suppose that tex2html_wrap_inline482 . It will not be necessary to assume that the transition function P is determined uniquely by Q.

We assume only that P is irreducible, that is, tex2html_wrap_inline490 for all tex2html_wrap_inline492 , and transient, that is,

displaymath494

LIMITING CONDITIONAL DISTRIBUTIONS

We shall be concerned with the existence of a limiting conditional distribution (LCD) for the process:

displaymath496

The conditional probability can be evaluated as follows:

equation57

where

displaymath498

Note that tex2html_wrap_inline500 and (by irreducibility and transience) tex2html_wrap_inline502 , tex2html_wrap_inline504 .

LCDs for ABSORBING CHAINS

Compare the above definition with the widely studied LCDs for absorbing chains:

Append to S an absorbing state (or, more generally, an absorbing set) tex2html_wrap_inline508 such that tex2html_wrap_inline510 for some (and then all) tex2html_wrap_inline504 and t>0.

In this context we study the limit (as tex2html_wrap_inline516 ) of

equation75

or, more generally (when tex2html_wrap_inline518 , the probability of absorption starting in j, is less than 1),

equation83

HITTING PROBABILITIES

Let tex2html_wrap_inline522 , so that tex2html_wrap_inline524 . This can be evaluated as

displaymath526

Claim. tex2html_wrap_inline528 , tex2html_wrap_inline504 , tex2html_wrap_inline532 .

It follows that tex2html_wrap_inline534 is a subinvariant vector for P; in particular,

displaymath538

where tex2html_wrap_inline540 .

Claim. b is a subinvariant vector for Q; in fact, b is `almost invariant':

displaymath548

CONSTRUCTION OF THE DUAL

Define tex2html_wrap_inline550 and tex2html_wrap_inline552 by

displaymath554

and

displaymath556

Clearly tex2html_wrap_inline558 is a (standard) transition function and tex2html_wrap_inline560 is a q-matrix such that tex2html_wrap_inline564 . However, P is dishonest with

displaymath568

and tex2html_wrap_inline560 is non-conservative with

displaymath572

(Recall that tex2html_wrap_inline500 .)

CONSTRUCTION OF THE DUAL

Append to S an absorbing state tex2html_wrap_inline508 and extend the definition of tex2html_wrap_inline558 and tex2html_wrap_inline560 to tex2html_wrap_inline584 as follows:

gather152

and

gather159

In this way tex2html_wrap_inline560 will be conservative and tex2html_wrap_inline558 will be honest.

tex2html_wrap_inline558 is a tex2html_wrap_inline560 -transition function since, for tex2html_wrap_inline504 ,

align165

INTERPRETATION OF THE DUAL

Let L be the last exit time of state 0, that is,

displaymath598

and define tex2html_wrap_inline600 by

displaymath602

tex2html_wrap_inline604 is tex2html_wrap_inline606 killed at time L. Thengif, since tex2html_wrap_inline610 , tex2html_wrap_inline604 is Markovian with transition function tex2html_wrap_inline558 , and, if tex2html_wrap_inline616 , its initial distribution is given by

displaymath618

Notice that tex2html_wrap_inline604 is absorbed with probability 1:

displaymath622

EXISTENCE OF LCDs

By the definition of tex2html_wrap_inline558 , we see immediately that, for every tex2html_wrap_inline492 ,

equation211

tex2html_wrap918

Here is one such result for absorbing chains:

Theorem. Suppose S is tex2html_wrap_inline630 -positive recurrent and let tex2html_wrap_inline632 be the (essentially unique) tex2html_wrap_inline630 -invariant measure for tex2html_wrap_inline558 . Then, if tex2html_wrap_inline638 ,

displaymath640

tex2html_wrap_inline630 -CLASSIFICATION

tex2html_wrap_inline558 and P have the same tex2html_wrap_inline630 -classification.

The decay parameter:

displaymath650

Note that tex2html_wrap_inline630 ( tex2html_wrap_inline654 ) is the same for each tex2html_wrap_inline492 and tex2html_wrap_inline658 , tex2html_wrap_inline660 .

Geometric ergodicity: tex2html_wrap_inline662 .

tex2html_wrap_inline630 -transience/recurrence: for tex2html_wrap_inline492 ,

displaymath668

tex2html_wrap_inline630 -positive/null recurrence: for tex2html_wrap_inline492 ,

displaymath674

tex2html_wrap_inline630 -INVARIANT MEASURES FOR P

Definition. A collection of positive numbers tex2html_wrap_inline680 is called a tex2html_wrap_inline630 -subinvariant measure for P if, for all tex2html_wrap_inline532 ,

displaymath688

and tex2html_wrap_inline630 -invariant if, for all tex2html_wrap_inline532 ,

displaymath694

Now, for any given collection tex2html_wrap_inline680 , define tex2html_wrap_inline632 by tex2html_wrap_inline700 .

Claim. m is tex2html_wrap_inline630 -(sub)invariant for P if and only if tex2html_wrap_inline708 is tex2html_wrap_inline630 -(sub)invariant for tex2html_wrap_inline558 .

Claim. If m is a tex2html_wrap_inline630 -subinvariant measure for P, then tex2html_wrap_inline720 if tex2html_wrap_inline722 . If m is tex2html_wrap_inline630 -invariant for P, then tex2html_wrap_inline720 only if tex2html_wrap_inline722 .

tex2html_wrap920

tex2html_wrap_inline630 -INVARIANT MEASURES FOR Q

Definition. A collection of positive numbers tex2html_wrap_inline680 is called a tex2html_wrap_inline630 -invariant measure for Q if, for all tex2html_wrap_inline532 ,

displaymath752

Again we have that m is tex2html_wrap_inline630 -invariant for Q if and only if tex2html_wrap_inline708 is tex2html_wrap_inline630 -invariant for  tex2html_wrap_inline560 .

Now suppose that Q is regular, so that P, now being the unique Q-transition function, is honest.

We can then apply to tex2html_wrap_inline560 the following result for absorbing chains (with a transient class S and absorbing state tex2html_wrap_inline508 ):

Theorem. If tex2html_wrap_inline632 is a finite tex2html_wrap_inline630 -invariant measure for tex2html_wrap_inline560 , then it is tex2html_wrap_inline630 -invariant for tex2html_wrap_inline558 if and only if

displaymath788

LCDs WHEN Q IS REGULAR

Theorem. Let tex2html_wrap_inline722 and suppose that tex2html_wrap_inline680 is a tex2html_wrap_inline630 -invariant measure for Q. Then, tex2html_wrap_inline800 , and m is tex2html_wrap_inline630 -invariant for P if (and only if)

displaymath808

whence if S is tex2html_wrap_inline630 -positive the LCD exists:

equation322

Other examples:

  1. Kesten's bounded jump conditionsgif.

  2. Asymptotic remotenessgif.

  3. Results for special processes: birth-death processes, branching processes, catastrophe processes.

BIRTH-DEATH PROCESSES

Consider an irreducible birth-death process on tex2html_wrap_inline458 . Its q-matrix is given by

displaymath820

where tex2html_wrap_inline822 ( tex2html_wrap_inline824 ), tex2html_wrap_inline826 ( tex2html_wrap_inline828 ) and tex2html_wrap_inline830 .

Define series A and C by

displaymath836

where tex2html_wrap_inline838 and, for tex2html_wrap_inline828 ,

displaymath842

Assume that tex2html_wrap_inline844 (Q regular) and that tex2html_wrap_inline848 (S transient). Then, the hitting probabilities are given by

displaymath852

Define polynomials tex2html_wrap_inline854 , where tex2html_wrap_inline856 , by tex2html_wrap_inline858 , tex2html_wrap_inline860 and, for tex2html_wrap_inline828 , tex2html_wrap_inline864 , and let tex2html_wrap_inline866 , tex2html_wrap_inline504 , where tex2html_wrap_inline630 is the decay parameter of S. Then, m is the essentially unique tex2html_wrap_inline630 -invariant measure.

The dual is an absorbing birth-death process; we get (in an obvious notation):

gather377

A direct application of van Doorn'sgif result for absorbing birth-death processes shows that the LCD exists if and only if tex2html_wrap_inline722 :

equation385

AN EXAMPLE

Let tex2html_wrap_inline880 , V>0 and tex2html_wrap_inline884 , and define tex2html_wrap_inline886 ,

displaymath888

where tex2html_wrap_inline890 .

This defines a density-dependent birth-death process with constant jump probabilities: think of V as ``area'' and n/V as ``population density''. The idea is that, as V gets large, the density process tex2html_wrap_inline898 becomes ``more deterministic'':

displaymath900

Clearly the process is irreducible if f(n)>0 for all tex2html_wrap_inline904 , and then transient if (and only if) tex2html_wrap_inline906 , in which case

displaymath908

Two choices for f:

displaymath912

Each has tex2html_wrap_inline914 and tex2html_wrap_inline916 .

Fig.1: Simulation

Fig.2: Limiting conditional distribution

Fig.3: Simulation

Fig.4: Limiting conditional distribution

Fig.5: Simulation

Fig.6: Limiting conditional distribution

Fig.7: Simulation

Fig.8: Limiting conditional distribution