1
$\begingroup$

I have the following general question, which I am not asking for help with here, because it is too general. I give some specific examples so that my motivation for asking might make more sense.

To what extent is there a formal/mathematical analogy between Bayesian updating/inference and Markov chains?

I am uncertain of how to find the answer for this question in the literature/in references about statistics, because I do not know what terms to search for. Nevertheless, I am sure it has a very well known answer, due to the extensive amount of research done on all of these topics.

Question: What key phrases would best be used to identify literature discussing the formal analogy between Bayesian "updating" and Markov chains?

I.e., what literature is there about the topic and how/where can one find it?


Examples of specific questions about the possible existence of a "formal"/mathematical analogy between the Bayesian update process for i.i.d. data and the convergence of a Markov Chain to a stationary distribution:

  • Is such an analogy the basis for the proof of the asymptotic correctness of MCMC?
  • Is the transition kernel between two different states of a Markov chain related to how one uses the likelihoods to transition between two different posteriors of a Bayesian updating process?
  • Are conjugate priors (for given likelihoods) related to/analogous to stationary starting distriubtions of a Markov chain (for given transition kernels)?
  • More generally, are priors analogous to the starting distribution of a Markov chain? (Initially important, but eventually unimportant as the Bayesian update process/Markov chain converges to its "final" state, which is independent of the prior/starting distribution of the MC.)
  • Is the detailed balance condition, $\pi(x)Q(y|x) = \pi(y)Q(x|y)$, which is sufficient for the existence of a unique stationary distribution, related to a symmetry condition for KL-divergence? I.e. in general $KL(p_x||p_y) \not= KL(p_y||p_x)$, just like how in general $\pi(x)Q(y|x) \not= \pi(y)Q(x|y)$. Also KL-divergence is related to results about the convergence of the posterior distribution to the true empirical distribution, the same way the symmetry condition in detailed balance guarantees convergence of the MC to a stationary distribution.

I.e., if we have a "linear" Markov chain, $Y_0, Y_1, \dots, Y_n, \dots$, and if we have a Bayesian update process for the distribution on the parameter space, $\lambda(\theta),$ $\lambda(\theta|X_1)$, $\lambda(\theta|X_1, X_2)$, $\dots$, $\lambda(\theta|X_1, \dots, X_n), \dots$, where the $X_i$ are i.i.d., then is it true that:
- The starting distribution, $\pi_0$ of $(Y_0, \dots, )$ is analogous to the prior $\lambda(\theta)$?
- The likelihoods $p(x|\theta)$ are analogous to the transition kernels $Q(x|y)$?
- The later MC states, $Y_1, \dots, Y_n, \dots,$ are analogous to the posteriors?
- The convergence to a stationary distribution of the MC is analogous to the convergence of the posteriors to the true distribution of the data?
- Asymmetry in the transition kernels is related to asymmetry in the KL-divergence of posteriors?

$\endgroup$
0
$\begingroup$

My professor said that when one fixes the value of the parameter, and then watch the posterior distribution after 0, 1, 2, ... observations, the result is a Markov chain.

(I.e. it is not even an analogy, but an exact correspondence in some sense.)

They also said that, however, for i.i.d. observations, the above scenario often does not satisfy the requirements for convergence to a stationary distribution.

Therefore, the Bayesian update process is in some sense not really analogous to Monte Carlo Markov chains which are always set up/rigged so that they converge to a stationary distribution (the posterior in the Bayesian update process after one update).

My professor gave me the following example to illustrate their point:

Suppose $X_1, \dots, X_n \sim Bern(\theta)$ and with a $Beta(\alpha,\beta)$ prior on $\theta$. The posterior after $n$ updates, with $S_n := \sum_{i=1}^n X_i$ is distributed as $Beta(\alpha + S_n, \beta + n - S_n)$.

Writing $(\alpha_n, \beta_n) = (\alpha_n + S_n, \beta + n - S_n)$, then the Bayesian update process gives a Markov chain on the state space $(\alpha_n, \beta_n)$, with transition kernel $Q(\alpha_{n+1}, \beta_{n+1}|\alpha_n, \beta_n)$ such that:

$$Q(\alpha_n +1, \beta_n|\alpha_n, \beta_n) = \theta $$ $$Q(\alpha_n, \beta_n + 1|\alpha_n, \beta_n) = 1 -\theta $$

Since $\alpha_n + \beta_n$ always increases by $1$, this Markov chain cannot have a stationary distribution. But we do have that $\dfrac{\alpha_n}{(\alpha_n + \beta_n)} \to \theta$, so that the posterior distribution will asymptotically concentrate around $\theta$.

If the prior satisfies Cromwell's rule, then by the argument found here, for the true value $\theta_0$ and any other value $\theta_1$, we have that $$\dfrac{\lambda(\theta_0 | X_1, \dots, X_n )}{\lambda(\theta_1 | X_1, \dots, X_n)} \to \infty $$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.