Given a probability measure $p$ on $\{1,\dots,n\}$ assumed to be the invariant measure of some irreducible ergodic Markov chain with unknown transition matrix $P$, i.e., $p = pP$, what (if any) problems about inferring a unique/optimal $P$ have been considered?

Examples of side conditions that might be in the literature somewhere to adequately specify a solution: requiring $P$ to be as sparse as possible along with some other condition like a few specific entries are known, or the sparsity pattern itself is completely known, or that $P$ is the exponential of some sparse (obviously embeddable) generator matrix, etc.

NB. I am aware that there are many stochastic matrices with a given invariant measure, that the invertible ones form a Lie semigroup, etc. Generic answers along these lines (i.e., that do not have some pointer to the literature or an actual special case that's been considered or solved or some such) are not helpful.

  • 1
    $\begingroup$ Welcome to our site. Generic questions like this one invite generic answers, so if you could edit your post to make it more specific or focused on a particular problem, that would help you get specific answers. $\endgroup$ – whuber Aug 23 '18 at 19:14
  • $\begingroup$ Interesting question, but I'm having trouble with motivation for it. MCMC is a classic example of constructing a Markov chain that converges to a known $p$. Often, the transition matrix is nearly sparse, in that a few entries tend to be way bigger than the rest. Yet, even if the matrix was sparse, it's hard to think of what advantage it would give. Doing iterations and finding eigenvalues (to infer convergence speed) would be faster. Yet, there's no reason for a sparse matrix to converge faster to the stationary distribution, which would likely be the whole point. $\endgroup$ – Alex R. Aug 31 '18 at 8:54
  • $\begingroup$ @AlexR.Here's one motivation: suppose $p$ is the concentration of metabolites and $P$ encodes metabolic pathways. You know a few of the reactions but not all, and sparsity is a reasonable Ansatz. More generally, suppose you can justify an Ansatz for sparse Markovian dynamics of some system but you can only measure the measure, and you want to know what the dynamics are. $\endgroup$ – S Huntsman Aug 31 '18 at 12:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.