Given the model $x_1,...,x_n$ ~ $Poi(\theta)$ (iid), and the prior $\theta$~$ Gamma(a,b)$. How do we prove the consistency of posterior distribution? Thanks in advance.

Modification: The definition of posterior consistency given on my lecture slides is:

“given the data $x_1,...,x_n$ iid and the model f(x;$\theta$), if we have posterior consistency at $\theta_0$, then $\forall \epsilon$, $\lim_{n\rightarrow +\infty}\pi(|\theta-\theta_0|>\epsilon|X)$=0.” $\theta_0$ is not specified but I think it is the true value of $\theta$.

  • $\begingroup$ Could you explain what you mean for a distribution to be consistent? $\endgroup$ – whuber Dec 22 '17 at 19:34
  • 1
    $\begingroup$ I have added this in the original question. $\endgroup$ – Maxxx. Dec 22 '17 at 20:06
  • 1
    $\begingroup$ Thank you. That was important, because consistency is not a property of any distribution: it's a property of the estimator and the model. (Presumably, "$\pi$" means "probability.") As far as your question goes, could you please indicate what attempts you have made towards a solution? For instance, have you worked out what the posterior distribution actually is in terms of $a$, $b$, and $X$? $\endgroup$ – whuber Dec 22 '17 at 20:48

Writing $X^{(n)}=(X_1,\dots,X_n)$, Bayes theorem tells us that $$ \Theta\mid X^{(n)}\sim\mathrm{Ga}\!\left(a+\sum_{i=1}^n X_i,\,b+n\right). $$ Define $$ M=\mathrm{E}[\Theta\mid X^{(n)}]=\frac{a+\sum_{i=1}^n X_i}{b+n} $$ and $$ V=\mathrm{Var}[\Theta\mid X^{(n)}]=\frac{a+\sum_{i=1}^n X_i}{(b+n)^2}. $$ For any $\epsilon>0$, using the triangle inequality we have $$ \epsilon \leq |\Theta-\theta_0| = |\Theta-M+M-\theta_0|\leq |\Theta-M|+|M-\theta_0|. $$ Hence, monotonicity and Chebyshev's (Чебышёв) inequality give us $$ \Pr(|\Theta-\theta_0|\geq\epsilon\mid X^{(n)}) \leq \frac{4V}{\epsilon^2} + \Pr(|M-\theta_0|\geq\epsilon/2\mid X^{(n)}). $$ Let $\mu_0$ be a $\mathrm{Poi}(\theta_0)$ distribution and note that $V\to 0$ a.s. $[\mu_0]$, and $M\to \theta_0$ a.s. $[\mu_0]$, as $n\to\infty$.

This is essentially what you need to see that this posterior is "simulation consistent" (that is, if you're simulating iid $X_i$'s from a $\mathrm{Poi}(\theta_0)$ distribution, then your posterior distribution will concentrate all its mass at $\theta_0$ as your sample size becomes arbitrarily large).

To learn how to write a precise proof, please study Schervish (Theory of Statistics), section 7.4.1 (Consistency of Posterior Distributions).

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.