3
$\begingroup$

Given the model $x_1,...,x_n$ ~ $Poi(\theta)$ (iid), and the prior $\theta$~$ Gamma(a,b)$. How do we prove the consistency of posterior distribution? Thanks in advance.

Modification: The definition of posterior consistency given on my lecture slides is:

“given the data $x_1,...,x_n$ iid and the model f(x;$\theta$), if we have posterior consistency at $\theta_0$, then $\forall \epsilon$, $\lim_{n\rightarrow +\infty}\pi(|\theta-\theta_0|>\epsilon|X)$=0.” $\theta_0$ is not specified but I think it is the true value of $\theta$.

$\endgroup$
3
  • 1
    $\begingroup$ Could you explain what you mean for a distribution to be consistent? $\endgroup$
    – whuber
    Commented Dec 22, 2017 at 19:34
  • 1
    $\begingroup$ I have added this in the original question. $\endgroup$
    – Maxxx.
    Commented Dec 22, 2017 at 20:06
  • 2
    $\begingroup$ Thank you. That was important, because consistency is not a property of any distribution: it's a property of the estimator and the model. (Presumably, "$\pi$" means "probability.") As far as your question goes, could you please indicate what attempts you have made towards a solution? For instance, have you worked out what the posterior distribution actually is in terms of $a$, $b$, and $X$? $\endgroup$
    – whuber
    Commented Dec 22, 2017 at 20:48

1 Answer 1

5
$\begingroup$

Writing $X^{(n)}=(X_1,\dots,X_n)$, Bayes theorem tells us that $$ \Theta\mid X^{(n)}\sim\mathrm{Ga}\!\left(a+\sum_{i=1}^n X_i,\,b+n\right). $$ Define $$ M=\mathrm{E}[\Theta\mid X^{(n)}]=\frac{a+\sum_{i=1}^n X_i}{b+n} $$ and $$ V=\mathrm{Var}[\Theta\mid X^{(n)}]=\frac{a+\sum_{i=1}^n X_i}{(b+n)^2}. $$ For any given $\epsilon>0$, if $|\Theta-M|<\epsilon/2$ and $|M-\theta_0|<\epsilon/2$, then, using the triangle inequality, we have $$ |\Theta-\theta_0| = |\Theta-M+M-\theta_0|\leq |\Theta-M|+|M-\theta_0| < \epsilon. $$ Considering the contrapositive of the previous statement, we see that $$ \{|\Theta-\theta_0|\geq\epsilon\} \subset \{|\Theta-M|\geq\epsilon/2\} \cup \{|M-\theta_0|\geq\epsilon/2\}. $$ Hence, monotonicity and Chebyshev's (Чебышёв) inequality give us $$ \Pr(|\Theta-\theta_0|\geq\epsilon\mid X^{(n)}) \leq \frac{4V}{\epsilon^2} + \Pr(|M-\theta_0|\geq\epsilon/2\mid X^{(n)}). $$ Let $\mu_0$ be a $\mathrm{Poi}(\theta_0)$ distribution and note that $V\to 0$ a.s. $[\mu_0]$, and $M\to \theta_0$ a.s. $[\mu_0]$, as $n\to\infty$.

This is essentially what you need to see that this posterior is "simulation consistent". That is, if you're simulating iid $X_i$'s from a $\mathrm{Poi}(\theta_0)$ distribution, then your posterior distribution will concentrate all its mass at $\theta_0$, as your sample size goes to infinity.

To learn how to write a precise proof, please study Schervish (Theory of Statistics), section 7.4.1 (Consistency of Posterior Distributions).

$\endgroup$
1

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.