Suppose we have $n$ samples ($x_1, x_2, …, x_n$) independently taken from a normal distribution, where known variance $\sigma^2$ and unknown mean $\mu$.

Considering non-informative prior distributions, the posterior distribution of the mean $p(\mu/D)$ follows normal distribution with $\mu_n$ and $\sigma_n^2$, where $\mu_n$ is the sample mean of the $n$ samples (i.e., $\mu_n=(x_1+x_2+…+x_n)/n)$, $\sigma_n^2$ is $\sigma^2/n$, and $D = \{x_1, x_2, …, x_n\}$ (i.e., $p(\mu/D) \thicksim N(\mu_n, \sigma^2/n)$).

Let the new data $D’$ be $\{x_1, x_2, …, x_n, x_1^{new}, x_2^{new}, …, x_k^{new}\}$. That is, we take additional $k$ ($k<n$) samples independently from the original distribution $N(μ, σ^2)$. However, before taking the additional samples, we can know the posterior predictive distribution for the additional sample. According to Bayesian statistics, the posterior predictive distribution $p(x^{new}/D)$ follows normal distribution with $\mu_n$ and $\sigma_n^2 + \sigma^2$ (i.e., $p(x^{new}/D) \thicksim N(\mu_n, \sigma_n^2 + \sigma^2)$). Namely, the variance becomes higher to reflect the uncertainty of $μ$. So far, this is what I know.

My question is, if we know $p(x^{new}/D)$ for the additional samples, can we predict the posterior distribution $p(μ/D')$ before taking the additional $k$ samples? I think that $p(μ/D’)$ seems to be calculated based on $p(x^{new}/D)$, but I have not gotten the answer yet. So, I need help. Please borrow your wisdom. Thanks in advance.

  • $\begingroup$ Good question. I have thought about this problem before and it's tricky. You're actually trying to calculate $p( p(\mu \vert D') \vert D)$, so a measure on possible posterior distributions if you were to collect more data. Instead, it might be easier to think about a summary measure of this future posterior distribution such as the future posterior mean, $\tilde{\mu}$, and try to calculate $p(\tilde{\mu} \vert D)$ $\endgroup$ – jsk May 3 '18 at 18:56
  • $\begingroup$ @jsk Thank for your constructive advise. But, i don't understand the differences between $p(\tilde{\mu}|D)$ and $p(\mu|D)$. Please could you tell me about this in detail? or suggest any reference that i can see? thanks for your consideration! $\endgroup$ – gigohan01 May 4 '18 at 6:49
  • $\begingroup$ I have no references. It's an idea I came up with in grad school, but never published any results. $p(\mu\vert D)$ is the posterior distribution of the parameter $\mu$ given the data. $p(\tilde{\mu} \vert D)$ is the posterior predictive distribution of the future parameter estimate given the current data. $\endgroup$ – jsk May 7 '18 at 19:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.