3
$\begingroup$

I have a group of samples drawn from density function $p(x,y)$, so it has the marginal density $p(x)$ and $p(y)$, and conditional density $p(x|y)$ and $p(y|x)$.

In what way I can construct another group of samples, with the SAME $p(x,y)$ but the $p(x)$, $p(y)$, $p(x|y)$ and $p(y|x)$ are changed?

$\endgroup$
  • 3
    $\begingroup$ The joint density determines the marginal densities and the conditional densities uniquely, and so the answer is No, you cannot construct another group of samples as you desire. If the joint density is being estimated from the samples instead of being known a priori, then different samples from the population will typically give slightly different estimates, and so the estimated $p(x,y)$ will be different as will marginals etc. $\endgroup$ – Dilip Sarwate Apr 5 '12 at 2:55
  • $\begingroup$ Thanks Dilip, my original thought is that $p(x,y) = p(x|y) p(y)$, so if you can "properly arrange" that decomposition, maybe we can get another $p(x,y)$ with different $p(x|y)$ and $p(y)$. Why it is not possible? $\endgroup$ – Lamfeeling Apr 5 '12 at 5:07
  • $\begingroup$ @DilipSarwate, convert your comment to answer. $\endgroup$ – mpiktas Apr 5 '12 at 8:36
  • $\begingroup$ @Lamfeeling It is not possible to "properly arrange" the decomposition since there is no choice in what we can use for $p(y)$: the joint density tells you what $p(y)$ must be, and then $p(x|y)$ is just the ratio $p(x,y)/p(y)$. $\endgroup$ – Dilip Sarwate Apr 5 '12 at 17:45
8
$\begingroup$

The joint density uniquely determines the marginal densities:

$$p(x) = \sum_y p(x,y) ~~\text{or}~~ p(x) = \int_{-\infty}^{\infty}p(x,y)\,\mathrm dy$$ (similarly for $p(y)$) and so the conditional densities are also determined uniquely by the joint density. So the answer is

No, you cannot construct another group of samples as you desire.

If the joint density of a population is being estimated from the samples instead of being known a priori, then different samples from the population will typically give slightly different estimates, and so the estimated $p(x,y)$ will be slightly different for the two sets of samples, as will marginals etc. Again, your desire to have the same joint density and different marginal densities will not be satisfied.

What you can get is the same marginal densities but different joint densities. A simple example is two Bernoulli random variables $X$ and $Y$ each with parameter $\frac{1}{2}$.

  • If they are independent, $p(0,0)=p(0,1)=p(1,0)=p(1,1) = \frac{1}{4}$.

  • But if $X = 1-Y$, $~p(0,0)=p(1,1)= 0, ~p(1,0)=p(0,1) = \frac{1}{2}$.

Consider also two standard normal random variables. If they are jointly normal, then $p(x,y)$ is the bivariate normal density. As a special case, if they are independent, then $p(x,y) = p(x)p(y)$. But they could be marginally normal random variables that are not jointly normal with joint density $$p(x,y) = \begin{cases} 2p(x)p(y), & \text{if}~ x \geq 0, y \geq 0, \text{or}~ x < 0, y < 0,\\ 0,& \text{otherwise}.\end{cases}$$ Note that the random variables are positively correlated in this case since the probability mass lies entirely in the first and third quadrants. This again illustrates that the same marginal densities can give rise to different joint densities.

What you want, the same joint density but different marginal densities, is, alas, not possible.

$\endgroup$
  • 1
    $\begingroup$ (+1) Good overview and nice, illustrative counterexample. It is interesting that if we sample from $p(x,y)$ we can recover a pair of iid standard normals $X^\star$ and $Y^\star$ by letting $X^\star = \epsilon_1 X$ and $Y^\star = \epsilon_2 Y$ where $\epsilon_i$ are independent random variables with $\mathbb P(\epsilon_i = \pm 1) = 1/2$. $\endgroup$ – cardinal Apr 5 '12 at 16:32
  • $\begingroup$ @cardinal To recover iid standard normals can we simply use e.g. $(X, Y^*)$ rather than $(X^*, Y^*)$? It seems to me that we don't need both of $\epsilon_1$ and $\epsilon_2$ to disentangle $X$ and $Y$. $\endgroup$ – Silverfish Dec 4 '14 at 14:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.