0
$\begingroup$

Suppose I have a parameter $\mu$ with a normal prior distribution with mean $\mu_0$ and standard deviation $\sigma_0$. On learning data $D$ with normal likelihood, with empirical mean $\mu_D$ and empirical standard deviation $\sigma_D$, the posterior is a normal distribution with mean $\mu'$ and standard deviation $\sigma'$ (which can be found in terms of $\mu_0,\sigma_0,\mu_D,\sigma_D$).

I am interested in a (weighted) sum of the prior and posterior, which is normally distributed. To find the standard deviation, we need the covariance of prior and posterior. Presumably, these are entirely dependent with covariance $1$, but I am unsure if that's right and if so I am unsure how to prove it.

$\endgroup$
2
  • $\begingroup$ I'm getting a strong x/y problem vibe out of this. Could you elaborate why are you looking for that weighted sum? $\endgroup$
    – jkm
    Dec 19, 2019 at 12:49
  • $\begingroup$ I'm doing Bayesian inference with uncertain data. So rather than updating to the normal posterior via Bayes' rule, I'm using something like Jeffrey's rule. I'm interested in the sum of prior and posterior, weighted by the probability that the data is correct $\endgroup$
    – DM-97
    Dec 19, 2019 at 13:00

1 Answer 1

1
$\begingroup$

This question does not seem to make sense from a probabilistic viewpoint. Both the prior and the posterior distributions are covering the same random variable, $\theta$ say, the prior being the marginal distribution of $\theta$ and the posterior the conditional distribution of $\theta$. It is thus impossible to consider a linear combination

of the prior and posterior, which is normally distributed

since, again, this is the same random variable.

$\endgroup$
3
  • 1
    $\begingroup$ yeah, thanks. what I originally wrote was clearly quite confused. What I mean is that I want a pdf given by a weighted sum of the prior ($p$) and posterior (given data $D$): $\bar{p}(\theta)=wp(\theta)+(1-w)p(\theta|D)$. In this case, the prior and posterior are both normal distributions $\endgroup$
    – DM-97
    Dec 24, 2019 at 17:27
  • 1
    $\begingroup$ Then the mixture of prior and posterior (a) is not a Normal (b) does not involve a correlation coefficient and (c) has no direct meaning from a Bayesian point of view. $\endgroup$
    – Xi'an
    Dec 25, 2019 at 10:42
  • $\begingroup$ Thanks, yep, makes sense. Yes, although this isn't used in classical Bayesianism, it makes sense when one is uncertain about the evidence that one is updating on. (statweb.stanford.edu/~cgates/PERSI/papers/AltBayes.PDF) $\endgroup$
    – DM-97
    Dec 26, 2019 at 12:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.