I have the following general question, which I am not asking for help with here, because it is too general. I give some specific examples so that my motivation for asking might make more sense.
To what extent is there a formal/mathematical analogy between Bayesian updating/inference and Markov chains?
I am uncertain of how to find the answer for this question in the literature/in references about statistics, because I do not know what terms to search for. Nevertheless, I am sure it has a very well known answer, due to the extensive amount of research done on all of these topics.
Question: What key phrases would best be used to identify literature discussing the formal analogy between Bayesian "updating" and Markov chains?
I.e., what literature is there about the topic and how/where can one find it?
Examples of specific questions about the possible existence of a "formal"/mathematical analogy between the Bayesian update process for i.i.d. data and the convergence of a Markov Chain to a stationary distribution:
- Is such an analogy the basis for the proof of the asymptotic correctness of MCMC?
- Is the transition kernel between two different states of a Markov chain related to how one uses the likelihoods to transition between two different posteriors of a Bayesian updating process?
- Are conjugate priors (for given likelihoods) related to/analogous to stationary starting distriubtions of a Markov chain (for given transition kernels)?
- More generally, are priors analogous to the starting distribution of a Markov chain? (Initially important, but eventually unimportant as the Bayesian update process/Markov chain converges to its "final" state, which is independent of the prior/starting distribution of the MC.)
- Is the detailed balance condition, $\pi(x)Q(y|x) = \pi(y)Q(x|y)$, which is sufficient for the existence of a unique stationary distribution, related to a symmetry condition for KL-divergence? I.e. in general $KL(p_x||p_y) \not= KL(p_y||p_x)$, just like how in general $\pi(x)Q(y|x) \not= \pi(y)Q(x|y)$. Also KL-divergence is related to results about the convergence of the posterior distribution to the true empirical distribution, the same way the symmetry condition in detailed balance guarantees convergence of the MC to a stationary distribution.
I.e., if we have a "linear" Markov chain, $Y_0, Y_1, \dots, Y_n, \dots$, and if we have a Bayesian update process for the distribution on the parameter space, $\lambda(\theta),$ $\lambda(\theta|X_1)$, $\lambda(\theta|X_1, X_2)$, $\dots$, $\lambda(\theta|X_1, \dots, X_n), \dots$, where the $X_i$ are i.i.d., then is it true that:
- The starting distribution, $\pi_0$ of $(Y_0, \dots, )$ is analogous to the prior $\lambda(\theta)$?
- The likelihoods $p(x|\theta)$ are analogous to the transition kernels $Q(x|y)$?
- The later MC states, $Y_1, \dots, Y_n, \dots,$ are analogous to the posteriors?
- The convergence to a stationary distribution of the MC is analogous to the convergence of the posteriors to the true distribution of the data?
- Asymmetry in the transition kernels is related to asymmetry in the KL-divergence of posteriors?