Gravity-Type Interactive Markov Models—Part II: Lyapunov Stability of Steady States

In Part I of this paper (Smith and Hsieh, 1997) a programming formulation of steady states was developed for gravity-type interactive Markov chains in terms of their associated spatial-flow chains. These results are here applied to analyze the stability properties of interactive Markov chains. In particular, the objective function for this programming formulation is shown to constitute a Lyapunov function for an appropriately defined continuous-time version of spatial-flow chains. The Lyapunov stability properties of these spatial flows are then shown to yield corresponding stability properties for the continuous-time versions of interactive Markov chains. In particular, these processes always exhibit global convergence to steady states. Finally, it is shown that when steady states are unique, these convergence results are inherited by those interactive Markov chains that are 'sufficiently close' to their continuous-time versions.