4
$\begingroup$

I have a group of samples drawn from density function $p(x,y)$, so it has the marginal density $p(x)$ and $p(y)$, and conditional density $p(x|y)$ and $p(y|x)$.

In what way I can construct another group of samples, with the SAME $p(x,y)$ but the $p(x)$, $p(y)$, $p(x|y)$ and $p(y|x)$ are changed?

Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ Thanks Dilip, my original thought is that $p(x,y) = p(x|y) p(y)$, so if you can "properly arrange" that decomposition, maybe we can get another $p(x,y)$ with different $p(x|y)$ and $p(y)$. Why it is not possible? $\endgroup$
    – Lamfeeling
    Commented Apr 5, 2012 at 5:07
  • 1
    $\begingroup$ @Lamfeeling It is not possible to "properly arrange" the decomposition since there is no choice in what we can use for $p(y)$: the joint density tells you what $p(y)$ must be, and then $p(x|y)$ is just the ratio $p(x,y)/p(y)$. $\endgroup$
    – Dilip Sarwate
    Commented Apr 5, 2012 at 17:45

1 Answer 1

Reset to default
13
$\begingroup$

The joint density uniquely determines the marginal densities:

$$p(x) = \sum_y p(x,y) ~~\text{or}~~ p(x) = \int_{-\infty}^{\infty}p(x,y)\,\mathrm dy$$ (similarly for $p(y)$) and so the conditional densities are also determined uniquely by the joint density. So the answer is

No, you cannot construct another group of samples as you desire.

If the joint density of a population is being estimated from the samples instead of being known a priori, then different samples from the population will typically give slightly different estimates, and so the estimated $p(x,y)$ will be slightly different for the two sets of samples, as will marginals etc. Again, your desire to have the same joint density and different marginal densities will not be satisfied.

What you can get is the same marginal densities but different joint densities. A simple example is two Bernoulli random variables $X$ and $Y$ each with parameter $\frac{1}{2}$.

  • If they are independent, $p(0,0)=p(0,1)=p(1,0)=p(1,1) = \frac{1}{4}$.

  • But if $X = 1-Y$, $~p(0,0)=p(1,1)= 0, ~p(1,0)=p(0,1) = \frac{1}{2}$.

Consider also two standard normal random variables. If they are jointly normal, then $p(x,y)$ is the bivariate normal density. As a special case, if they are independent, then $p(x,y) = p(x)p(y)$. But they could be marginally normal random variables that are not jointly normal with joint density $$p(x,y) = \begin{cases} 2p(x)p(y), & \text{if}~ x \geq 0, y \geq 0, \text{or}~ x < 0, y < 0,\\ 0,& \text{otherwise}.\end{cases}$$ Note that the random variables are positively correlated in this case since the probability mass lies entirely in the first and third quadrants. This again illustrates that the same marginal densities can give rise to different joint densities.

What you want, the same joint density but different marginal densities, is, alas, not possible.

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ (+1) Good overview and nice, illustrative counterexample. It is interesting that if we sample from $p(x,y)$ we can recover a pair of iid standard normals $X^\star$ and $Y^\star$ by letting $X^\star = \epsilon_1 X$ and $Y^\star = \epsilon_2 Y$ where $\epsilon_i$ are independent random variables with $\mathbb P(\epsilon_i = \pm 1) = 1/2$. $\endgroup$
    – cardinal
    Commented Apr 5, 2012 at 16:32
  • $\begingroup$ @cardinal To recover iid standard normals can we simply use e.g. $(X, Y^*)$ rather than $(X^*, Y^*)$? It seems to me that we don't need both of $\epsilon_1$ and $\epsilon_2$ to disentangle $X$ and $Y$. $\endgroup$
    – Silverfish
    Commented Dec 4, 2014 at 14:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.