3
$\begingroup$

Given a sample vector $x$ of size $N$ from a normally distributed population. With frequentist methods the population mean is estimated as $\hat{\mu}=\frac{\Sigma{}x_i}{N}$, population sigma is estimated as $\hat{\sigma}=\sqrt{\frac{\Sigma{(x_i - \hat{\mu})^2}}{N - 1}}$.

Is there a Bayesian prior for the population mean and population sigma that will lead to the same estimates as the above frequentist formulas for any $x$?

Improve this question
$\endgroup$
1
  • 2
    $\begingroup$ $p(\mu,\sigma) \propto \sigma^{-1}, \,\, \mu \in \mathbb{R},\, \sigma \in (0, \infty)$ $\endgroup$
    – Cyan
    Mar 29, 2013 at 12:25

1 Answer 1

Reset to default
2
$\begingroup$

As pointed out by @Cyan in the comment above, the prior $p(\mu,\sigma) \propto \sigma^{-1}$ is a probability matching prior, in the following sense:

  1. Define $t=\frac{\mu-\bar{x}}{s/\sqrt{n}}$, where $\bar{x}$ and $s^2$ are the sample mean and variance (the same statistics you use as estimators).

  2. It follows that the posterior distribution of $t|(X_1,...,X_n)$ is student's t with $n-1$ degrees of freedom, precisely the distribution of $t$ as a pivotal quantity in the frequentist pardigm.

The implications are, for instance, that a HPD interval for $t$ coincides with the corresponding level confidence interval for $\mu$.

The joint posterior distribution for $(\mu, \sigma)$ is analytically intractable, but the posterior modes take the form:

$$ \mu = \bar{x} $$

$$ \sigma = \sqrt{\frac{n-1}{n+1}s^2} $$

Improve this answer
$\endgroup$
7
  • $\begingroup$ Do you mean that using the prior $p(\mu,\sigma) \propto \sigma^{-1}$ the posterior MEANS of $\mu$ and $\sigma$ will be equal to the above $\hat{\mu}$ and $\hat{\sigma}$? $\endgroup$
    – winerd
    Mar 29, 2013 at 16:08
  • $\begingroup$ @winerd, I'm working on it, I believe this is the case indeed. The posterior is $p(\mu,\sigma|\boldsymbol{x}) \propto \sigma^{-n-1} e^{{-((n-1)s+n(\bar{x}-\mu)^2)/(2\sigma^2)}}$. I'll update the answer as soon as I', done :-) $\endgroup$
    – Ferdinand.kraft
    Mar 29, 2013 at 17:06
  • $\begingroup$ @winerd, Not for the standard deviation parameter. What is true is that the posterior mean of $\sigma^2$ is $\hat{\sigma}^2$. (Similarly, $\hat{\sigma}^2$ is unbiased for $\sigma^2$, but $\hat{\sigma}$ is not unbiased for $\sigma$.) $\endgroup$
    – Cyan
    Mar 29, 2013 at 18:53
  • $\begingroup$ @winerd, after struggling a few days with the posterior I must admit it is analytically intractable, an mathematica agrees with me... The best I could do is to find the posterior modes. $\endgroup$
    – Ferdinand.kraft
    Apr 2, 2013 at 2:59
  • $\begingroup$ @Ferdinand.kraft, that's not correct. Check out this and notice that the math still goes through if $\alpha_0 = \beta_0 = \lambda_0 = 0$. $\endgroup$
    – Cyan
    Apr 2, 2013 at 6:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.