0
$\begingroup$

Say we have $n$ data points $x_1, x_2, ..., x_n$, let's also assume that each data point comes from a normal distribution. For Bayesian inference,

the first prior assumption is, for each data point $x_i$, $i=1, ..., n$, we assume $x_i \sim N(\mu_i,\sigma^2)$, also assume that $\sigma^2$ is known, so we only need to specify a prior on each $\mu_i$, we assume $\mu_i \sim N(\lambda_1,\lambda_2)$;

the second prior assumption is, for each data point $x_i$, $i=1, ..., n$, we assume $x_i \sim N(\mu,\sigma^2)$, $\sigma^2$ is known, we also assume $\mu \sim N(\lambda_1,\lambda_2)$;

apparently, if we do Bayesian inference for the posterior distribution. The two prior assumptions will give us two different results, for the first one, we have $n$ posterior distributions for each $\mu_i$, and for the second one, we only have one posterior distribution for $\mu$. However, what are the differences between the two prior assumptions, it seems like they are the same. And which one is more appropriate?

$\endgroup$
2
  • 1
    $\begingroup$ The first does not assume that the expectations are the same for each observation; the second assumes that the expectations are the same for each observation. So the more appropriate depends on your assumption. The second would allow you to narrow your distribution for further data points in the light of the evidence if they too were assumed to have the same expectation, but would be inappropriate if this was not the case $\endgroup$
    – Henry
    Commented Jul 5, 2021 at 8:22
  • 2
    $\begingroup$ In the first model, the parameters are independent a priori and a posteriori. In the second model, they are independent a priori and dependent a posteriori. $\endgroup$
    – Xi'an
    Commented Jul 5, 2021 at 10:09

1 Answer 1

1
$\begingroup$

The two assumptions are not identical. By using the theorem of total expectation (and of total variance) we use the assumption 1. to calculate the marginal distribution for $x_i$. It is normal, with expectation and variance given by: $$\DeclareMathOperator{\E}{\mathbb{E}} \DeclareMathOperator{\V}{\mathbb{V}} \E x_i = \E \E x_i \mid \mu_i = \E \mu = \lambda_1 \\ \V x_i = \E \V x_i \mid \mu_i + \V \E x_i \mid \mu_i = \\ \E \sigma^2 + \V \lambda_1 = \sigma^2 + \lambda_1 $$ which is not the same as given under assumption 2.

Se also the comments under the question.

To get equality you must replace assumption 2. by $$ x_i \sim N(\mu_i,\sigma^2+\lambda_1) $$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.