7
$\begingroup$

One of the assumptions in a model is the conditional dependence between random variables in the joint prior distribution. Consider the following model, $$p(a,b|X) \propto p(X|a,b)p(a,b)$$

Now suppose an independence assumption for the prior $p(a,b) = p(a)p(b)$.

Does this assumption imply the posterior has the following conditional dependence as well? $$p(a|X)p(b|X) \propto p(X|a,b)p(a)p(b)$$

$\endgroup$
  • $\begingroup$ If $p(X \mid a,b)=f(X\mid a)\,g(X\mid b)$ for some $f,g$ then perhaps this might give posterior independence $\endgroup$ – Henry Jun 21 at 14:35
18
$\begingroup$

Your question can also be stated as: "$X$ is dependent on $a$ and $b$. And $a$ and $b$ are independent. Does this imply that $a$ and $b$ are conditionally independent given $X$?"

The answer is no. We just need a counter-example to show it isn't the case. Suppose $X = a + b$.

Then, once we know $X$'s value, $a$ and $b$ are dependent (information about one tells us what the other will be). For example, suppose $X=5$. Then, if $a=3$, it tells us that $b=2$. Similarly, if $b=4$, it tells $a=1$.

$\endgroup$
10
$\begingroup$

No, it doesn't: Under the assumption that $a \ \bot \ b$, the right-hand-side of your last equation is:

$$p(x|a,b) \cdot p(a) \cdot p(b) = p(x,a,b) \overset{a,b}{\propto} p(a,b|x).$$

Thus, you are effectively asking whether or not:

$$a \ \bot \ b \quad \quad \implies \quad \quad p(a|x) \cdot p(b|x) \propto p(a,b|x).$$

That is, you are asking whether prior independence of $a$ and $b$ implies posterior independence of these random variables. Generally speaking, no it doesn't --- many statistical models involve data $x$ that give information about both prior variables, in such a way that they exhibit statistical dependence a posteriori.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.