0
$\begingroup$

Given is a Gaussian $\mathcal{N}(\mu,\sigma^2)$ with unknown $\mu$ and known $\sigma^2$. There is only one data point $x$. What is the parameter for the mean of the posterior distribution, i.e. what do I know about the real $\mu$?

The posterior mean after 1 observation is

$$\mu_1=x-(x-\mu_0)\frac{\sigma^2}{\sigma^2+\sigma_0^2}$$ with $\mu_0$ and $\sigma_0^2$ the mean and variance of the prior (eq.33 in this source). But which values shall be selected for $\mu_0$ and $\sigma_0$? Selecting $\mu_0=0$ and $\sigma_0=\infty$ gives the trivial solution $\mu_1=x$.

For example let's assume we know $\sigma^2=0.1$ and the data is $x=2$. What can we say about the location of $\mu$?

Improve this question
$\endgroup$
10
  • 1
    $\begingroup$ Won't the answer depend on the prior distribution? $\endgroup$
    – Michael R. Chernick
    Commented Dec 30, 2019 at 16:56
  • $\begingroup$ But which parameter shall be selected for the prior distribution to get a nontrivial result? If I measure once a variable where I know that it is Gaussian distributed and I know $\sigma^2$ of this distribution then what do I know about the real $\mu$? Maybe Bayesian thinking does not help in this case? $\endgroup$
    – granular_bastard
    Commented Dec 30, 2019 at 17:05
  • 2
    $\begingroup$ It is possible even without any prior assumptions to construct a bounded confidence interval for $\mu$ based on a single observation. See stats.stackexchange.com/a/1836/919. $\endgroup$
    – whuber ♦
    Commented Dec 30, 2019 at 17:13
  • 1
    $\begingroup$ My question is how do you get the formula for the posterior mean? Why have a prior variance other than the known value of $\sigma^2$. $\endgroup$
    – Michael R. Chernick
    Commented Dec 30, 2019 at 18:50
  • 1
    $\begingroup$ @Michael R. Chernick: Based on the link, it looks the prior is also normal ( see equation 12 in that ) which is where that equation comes from when you have one observation. I'm not clear on the question ? You ( not you but the OP ) have the prior and the result for the update, so what's the question ? To the OP: you have to be given $\mu_0$ and $\sigma^2_0$ of the prior in order to do the update. $\endgroup$
    – mlofton
    Commented Dec 30, 2019 at 19:17

1 Answer 1

Reset to default
2
$\begingroup$

If your prior $\sigma_0 = \infty$, this implies that your prior is totally uninformative. Therefore, given one single observation, your posterior belief about the mean is simply the observation itself, which you have already pointed out.

The posterior precision (inverse of variance) about the mean is simply the sum of the prior precision and the precision of the data. In this case, the posterior precision is $\frac{1}{\infty}+\frac{1}{0.1}$, so the posterior variance is the same as the variance of the data generating process. This is again due to the fact that the prior is uninformative.

If you had some informative prior beliefs with finite variance, the posterior mean is as you have described, where $\mu_1=\frac{\sigma^2}{\sigma^2+\sigma_0^2}\mu_0+ \frac{\sigma_0^2}{\sigma^2+\sigma_0^2}x $. This is just a rewriting of what you had, and we can interpret this as a weighted combination of the prior belief and the data. The weights are simply the variance or degree of uncertainty about each phase, thus if your prior is uninformative, we can interpret this as assigning all weight to the data and no weight to the prior.

The choice of the prior depends on the context and is important to justify in any Bayesian inference problem. One example is where you begin with an uninformative prior and observe some data, thus forming a posterior belief. This posterior belief can then form your “prior” going forward, which you will update based on the weighted combination formula above after you observe more data.

Improve this answer
$\endgroup$
3
  • $\begingroup$ This means the starting point was wrong as there is no useful prior. But the question still remains: What do I know about $\mu$ if I know that there is a Gaussian distribution with known $\sigma^2$ given a single measurement $x$? $\endgroup$
    – granular_bastard
    Commented Dec 31, 2019 at 17:32
  • $\begingroup$ it depends on what you want to assume. do you have a prior ? if not, then you probably want to go classical route. that can then be delived in further as to what to minimize when estimating $\mu$. OTOH, If you have do have a prior, then use posterior proportional to prior * likelihood where likelihood is nornal. So, there are two branches to head down and which depends on whether you have a prior. or to put it differently: if you believe in bayesian, then choose some non-informative prior. $\endgroup$
    – mlofton
    Commented Dec 31, 2019 at 23:00
  • $\begingroup$ and I said in my other comment, the MLE of $\mu$ is $x$ but that's classical viewpoint which maybe can modified to a bayesian setting with the right prior.Also,a great, thorough answer by bayes. thanks. $\endgroup$
    – mlofton
    Commented Dec 31, 2019 at 23:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.