Invited talk


Workshop celebrating Tony Pakes' 60th Birthday


Phil Pollett

The University of Queensland


Pakes, A.G. (1969) Some conditions for ergodicity and recurrence of Markov chains. Operat. Res. 17, 1058-1061.

Let tex2html_wrap_inline651 be an irreducible aperiodic Markov chain taking values in the non-negative integers and let


Then, tex2html_wrap_inline655 for all i sufficiently large is enough to guarantee recurrence, while tex2html_wrap_inline659 and tex2html_wrap_inline661 is sufficient for ergodicity.

This result has been used by many authors in a variety of contexts, for example, in the control of random access broadcast channels: slotted Aloha and CSMA/CD (Carrier sense multiple access with collision detect) protocol.

The Aloha Scheme

The following description is based on (Kelly, 1985)gif.

Several stations use the same channel (assume infinitely many stations). Packets arrive for transmission as a Poisson stream with rate tex2html_wrap_inline663 (<1). Time is broken down into ``slots" (0,1], (1,2], tex2html_wrap_inline671. Let tex2html_wrap_inline673 be the number of packets to arrive in the slot (t-1,t] ( tex2html_wrap_inline677 ). Their transmission will first be attempted in the next slot (t,t+1]. Let tex2html_wrap_inline681 represent the output of the channel at time t:


If tex2html_wrap_inline687, a ``collision'' has occurred, and retransmission will be attempted in later slots, independently in each slot with probability f until successful. Thus, the transmission delay (measured in slots) has a geometric distribution with parameter 1-f.

The backlog tex2html_wrap_inline693 is a Markov chain with






We deduce that tex2html_wrap_inline697 for all n sufficiently large. Indeed the chain is transient (Kleinrock (1983), Fayolle, Gelenbe and Labetoulle (1977), Rosenkrantz and Towsley (1983)).

State-dependent Retransmission

Now suppose that the retransmission probability is allowed to depend on the backlog: tex2html_wrap_inline701 when tex2html_wrap_inline703. Then, tex2html_wrap_inline705 is maximized by


and, with this choice,


Thus, tex2html_wrap_inline709 and tex2html_wrap_inline711. Thus, tex2html_wrap_inline693 is ergodic, that is, the backlog is eventually cleared, if tex2html_wrap_inline715.

But, users of the channel do not know the backlog, and thus cannot determine the optimal retransmission probability.

Towards a Better Control Scheme

It would be better to choose the retransmission probability tex2html_wrap_inline717 based on the observed channel output. Several schemes have been suggested by Mikhailov (1979) and Hajek and van Loon (1982). For example, suppose each station maintains a counter tex2html_wrap_inline719, updated as follows: tex2html_wrap_inline721 and


where a,b and c are to be specified. For example, (a,b,c) = (-1,0,1) is an obvious choice. Suppose that tex2html_wrap_inline729. Then, tex2html_wrap_inline731 is a Markov chain. We would like tex2html_wrap_inline719 to ``track'' the backlog, at least when tex2html_wrap_inline735 is large. Consider the drift in tex2html_wrap_inline737 :


Let tex2html_wrap_inline739 with tex2html_wrap_inline741 held fixed. Then,


The choice tex2html_wrap_inline745, where tex2html_wrap_inline747, makes the drift in tex2html_wrap_inline737 negative if tex2html_wrap_inline751 and positive if tex2html_wrap_inline753. Thus, if the backlog were held steady at a large value, then the counter would approach that value. Also,


Mikhailov (1979) showed that the choice (a,b,c) =(2-e,0,1) ensures that tex2html_wrap_inline731 is ergodic whenever tex2html_wrap_inline761.

Question. For an irreducible aperiodic Markov chain tex2html_wrap_inline731, can one infer anything about its ergodicity and recurrence from the marginal drifts?


Pakes, A.G. (1987) Limit theorems for the population size of a birth and death process allowing catastrophes. J. Math. Biol. 25, 307-325.

An appropriate model for populations that are subject to crashes (dramatic losses can occur in animal populations due to disease, food shortages, significant changes in climate).

Such populations can exhibit quasi-stationary behaviour: they may survive for long periods before extinction occurs and can settle down to an apparently stationary regime. This behaviour can be modelled using a limiting conditional (or quasi-stationary) distribution.

The Model

It is a continuous-time Markov chain tex2html_wrap_inline765, where X(t) represents the population size at time t, with transition rates tex2html_wrap_inline771 given by


with the other transition rates equal to 0. Here, tex2html_wrap_inline773, a>0 and tex2html_wrap_inline777 for at least one i in tex2html_wrap_inline781, and, tex2html_wrap_inline783.

Interpretation. For tex2html_wrap_inline785, tex2html_wrap_inline787 is the instantaneous rate at which the population size changes from j to k, tex2html_wrap_inline793 is the per capita rate of change and, given a change occurs, a is the probability that this results in a birth and  tex2html_wrap_inline797 is the probability that this results in a catastrophe of size i (corresponding to the death or emigration of i individuals).

Some Properties

The state space. Clearly 0 is an absorbing state (corresponding to population extinction) and C is an irreducible class.

Extinction probabilities. If tex2html_wrap_inline805 is the probability of extinction starting with i individuals, then tex2html_wrap_inline809 for all tex2html_wrap_inline811 if and only if D (the expected increment size), given by


is less than 0 (the subcritical case) or equal to 0 (the critical case).

In the supercritical case (D>0), the extinction probabilities can be expressed in terms of the probability generating function


We find that


where b(s)=f(s)-s.

Limiting Conditional Distributions

In order to describe the long-term behaviour of the process, we use two types of limiting conditional distribution (LCD), called Type I and Type II, corresponding to the limits:


where tex2html_wrap_inline825. Thus, we seek the limiting probability that the population size is j, given that extinction has not occurred, or (in the second case) will not occur in the distant future, but that eventually it will occur; we have conditioned on eventual extinction to deal with the supercritical case, where this event has probability less than 1.

The Existence of Limiting Conditional Distributionsgif

Consider the two eigenvector equations


where tex2html_wrap_inline829 and C is the irreducible class.

In order that both types of LCD exist, it is necessary that these equations have strictly positive solutions for some tex2html_wrap_inline833, these being the positive left and right eigenvectors of  tex2html_wrap_inline835 (the transition-rate matrix restricted to C) corresponding to a strictly negative eigenvalue  tex2html_wrap_inline839.

Let tex2html_wrap_inline841 be the maximum value of  tex2html_wrap_inline843 for which positive eigenvectors exist ( tex2html_wrap_inline841 is known to be finite), and denote the corresponding eigenvectors by tex2html_wrap_inline847 and tex2html_wrap_inline849.

The Existence of Limiting Conditional Distributions

Proposition.gif Suppose that Q is regular.

If tex2html_wrap_inline855 converges, and either tex2html_wrap_inline857 converges or tex2html_wrap_inline859 is bounded, then the Type II LCD exists and defines a proper probability distribution tex2html_wrap_inline861 over C, given by


(All unmarked sums are over k in C.)

If in addition tex2html_wrap_inline871 converges, then the Type I LCD exists and defines a proper probability distribution tex2html_wrap_inline873 over C, given by


Try to use PKP Technology

We need the fact that b(s)=0 has a unique solution  tex2html_wrap_inline881 on [0,1], and that tex2html_wrap_inline885 or tex2html_wrap_inline887 according as tex2html_wrap_inline889 or D<0.

Setting tex2html_wrap_inline893, the eigenvector equations can be written (for tex2html_wrap_inline895 ) as



What is the maximum value of tex2html_wrap_inline843 for which a positive solution exists? If tex2html_wrap_inline849 is any solution to the second, then its generating function tex2html_wrap_inline905 satisfies


where, for tex2html_wrap_inline909, tex2html_wrap_inline911.

Using this approach, we cannot really avoid the question: when is X(s) a power series with non-negative coefficients? The function tex2html_wrap_inline915 satisfies


So, equivalently, we ask: when does C(s) have non-negative coefficients?

This is answered in the following paper (assuming, as we have here, that B(s) is a power series with non-negative coefficients):

Pakes, A.G. (1997) On the recognition and structure of probability generating functions. In (Eds. K.B. Athreya and P. Jagers) Classical and Modern Branching Processes, IMA Vols. Math. Appl. 84, Springer, New York, pp. 263-284.

Lemon. The maximum value of tex2html_wrap_inline843 for which a positive right eigenvector exists is tex2html_wrap_inline925. When tex2html_wrap_inline927, the left eigenvector is given by tex2html_wrap_inline929, tex2html_wrap_inline895.

The Subcritical Case

We have D:=-b'(1-)<0 and tex2html_wrap_inline935. Since tex2html_wrap_inline929, tex2html_wrap_inline895, we have also tex2html_wrap_inline941 and tex2html_wrap_inline943.

The combination of technologies thus yields:

Theorem. In the subcritical case both types of LCD exist. The Type I LCD is given by


and the Type II LCD has pgf




and, for tex2html_wrap_inline909, tex2html_wrap_inline911.

This result is contained in Theorems 5.1 and 6.2 of Pakes (1987).

The Supercritical Case

We have D>0 and tex2html_wrap_inline885, and the absorption probabilities have generating function


Since tex2html_wrap_inline961, tex2html_wrap_inline895, we have tex2html_wrap_inline965 and tex2html_wrap_inline967. When do these series converge?

Condition (A). The catastrophe-size distribution has finite second moment, that is, tex2html_wrap_inline969 (equivalently tex2html_wrap_inline971 ).

Condition (B). The function b can be written


where L is slowly varying, that is, tex2html_wrap_inline979 for large t.

Theorem. In the supercritical case, the Type I LCD exists under (A), and is given by tex2html_wrap_inline983. If in addition (B) holds, then the Type II LCD exists and has pgf tex2html_wrap_inline985.

This first part (Type I LCD) is contained in:

Pakes, A.G. and Pollett, P.K. (1989) The supercritical birth, death and catastrophe process: limit theorems on the set of extinction. Stochastic Process. Appl. 32, 161-170.

The second part (Type II LCD) is contained in Theorem 6.2 of Pakes (1987).

Other papers important to my work:

Pakes, A.G. (1971) A branching process with a state dependent immigration component. Adv. Appl. Probab. 3, 301-314.

Pakes, A.G. (1975) On the tails of waiting-time distributions. J. Appl. Probab. 12, 555-564.

Pakes, A.G. (1992) Divergence rates for explosive birth processes. Stochastic Process. Appl. 41, 91-99.

Pakes, A.G. (1993) Explosive Markov branching processes: entrance laws and limiting behaviour. Adv. Appl. Probab. 25, 737-756.

Pakes, A.G. (1993) Absorbing Markov and branching processes with instantaneous resurrection. Stochastic Process. Appl. 48, 85-106.

Pakes, A.G. (1995) Quasi-stationary laws for Markov processes: examples of an always proximate absorbing state. Adv. Appl. Probab. 27, 120-145.