Given a random vector $X \in \mathbb{R}^k$, with a known pdf given by $f_X$. If $Y, Z \in \mathbb{R}^k$ are defined by $Y = AX$, $Z = BX$, where $A,B \in \mathbb{R}^{k\times k}$ are different, given, real-valued matrices.

I know how to calculate pdfs of $Y$ and $Z$ on their own. But how do I derive the joint pdf of $Y$ and $Z$?

If it helps to be more specific, $f_X$ is a mixture of $0$-mean multivariate gaussians, each component in the mixture with a different, diagonal covariance matrix (but not of the form $\Sigma = \sigma^2 I$).

Any help would be much appreciated.

For some context:

My goal is to check for the $f_X$ mentioned above, and a specific $A$ and $B$, whether the vectors $Y$ and $Z$ are independent. This means I need to check whether the joint distribution of $Y$ and $Z$ factorises into the product of the marginals. There are at least some cases when this is true: if, for example, $X \sim \mathcal{N}(0,\sigma^2 I)$ and $A$ and $B$ are projections onto orthogonal subspaces. But proving it is not true in my case would also be helpful. Hence my need to derive the joint distribution of $Y$ and $Z$.


Both $A$ and $B$ are square matrices, so there are a couple of basic points to note:

  • If $A$ is non-singular then $Z = BX = B (A^{-1} A X) = B A^{-1} Y$; and similarly
  • If $B$ is non-singular then $Y = AX = A (B^{-1} B X) = A B^{-1} Z$.

We can see from these results that if $A$ is non-singular then there is a function $Y \mapsto Z$ and if $B$ is non-singular then there is a function $Z \mapsto Y$. In either case the joint density of the two random vectors follows trivially from the marginal density of the first, and the random vectors will not be independent. (Indeed, one is a deterministic function of the other.)

This should cover almost all cases that you encounter. Indeed, it means that the only non-trivial case of interest is the case where both $A$ and $B$ are singular matrices. In this latter case things get trickier, and you will need to figure out the sets of values of one random vector that correspond to an individual value of the other random vector. Even in this case it is highly unlikely that you can obtain independence of the random vectors, since that would require special construction.

  • $\begingroup$ Firstly, thank you very much for the answer @Ben. I feel a bit silly for not immediately realising the points mentioned above. Nonetheless it turns out that my A and B are in fact both singular. My apologies for not realising and mentioning that important fact in the question. I will edit to clarify. I’m new to asking questions on SE so not sure whether it’s appropriate to edit the question in light of progress based on your answer. If not do let me know in a comment and I’ll post it again as a separate question. $\endgroup$ – h2learn Apr 17 '19 at 10:06
  • $\begingroup$ @Price: That's fine - no need to feel silly. In cases like this where there is already an answer that would be rendered redundant by new details, I think the best procedure is to ask a new question with the additionally specified details there. You can always draft your new question to link back to this one to show the history of your inquiry. $\endgroup$ – Ben Apr 17 '19 at 10:44
  • $\begingroup$ Ok, sure. Have changed back, will add a new question. $\endgroup$ – h2learn Apr 17 '19 at 10:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.