3
$\begingroup$

What is the (largest) set of matrices $\mathcal C\subset \mathbb R^{m\times n}$ ($m\le n$) for which the following statement is true?

Let $x_1,\ldots,x_n\in\mathbb R$ be independent random variables. Then also $z_1, \ldots, z_m$ are independent, given $z=Ax$ with $A\in\mathcal C$

It obviously holds when $A$ is a (rectangular) diagonal matrix. Is it also true if the rows are (1) orthogonal (2) linearly independent?

Conjecture: It is only true for matrices for which it is trivially true, i.e. when there is no interaction between any of the $x_i$: $$\mathcal C = \{DP\mid D\in\mathbb R^{m\times n} \text{ diagonal, } P\in\mathbb R^{n\times n} \text{ permutation matrix} \}$$

$\endgroup$
  • 1
    $\begingroup$ linear independent a definite no, think about $$A = \begin{bmatrix} 1 & 1\\ 0 & 1 \end{bmatrix}$$ $Ax = [X,X+Y]$ where $X,X+Y$ are definitely not independent. $\endgroup$ – David Veitch Feb 20 at 14:01
4
$\begingroup$

This is the set $C_{m,n}$ of matrices that have at most one nonzero entry in each column.

Let us rephrase the issue in the question as a property of an $m\times n$ matrix $A:$

$\mathcal{P}(A):$ for all independent random variables $x=(x_1,\ldots,x_n)$ defined on a given space $\Omega,$ $z=Ax$ is independent.

My assertion requires two demonstrations: namely, that $A\in C_{m,n}$ implies $\mathcal{P}(A)$ and then that $A\notin C_{m,n}$ implies $\mathcal{P}(A)$ is false.

First, when $A\in C_{m,n}$ and the $x_j$ are independent, we have to show the $z_i$ are independent. This can be done with an induction on $m.$ When $m=1,$ $\{z_1\}$ is trivially independent. Now assume the result for $m-1.$ The structure of $A$ is such that $z_m$ can be expressed in terms of the $x_j$ for which $A_{mj}\ne 0$ and all the other $z_i,$ $i\lt m,$ can be expressed in terms of the remaining $x_j.$ Because the $x_j$ are independent, $z_m$ is therefore independent of the remaining $z_i,$ whence $\{z_1,\ldots,z_m\}$ is independent. That establishes the first claim.

Second, suppose $A \notin C_{m,n}.$ This means there is a column $j$ and two rows $i,i^\prime$ for which $A_{ij} \ne 0$ and $A_{i^\prime j}\ne 0.$ We may therefore write

$$z_i = A_{ij} x_j + y_i,\quad z_{i^\prime} = A_{i^\prime j} x_j + y_{i^\prime}$$

for variables $y_i$ and $y_{i^\prime}$ formed out of the remaining $x_{j^\prime},$ $j^\prime\ne j.$ Setting all such $x_{j^\prime}$ to be constant and selecting $x_j$ to have unit variance, compute

$$\operatorname{Cov}(z_i,z_{i^\prime}) = A_{ij}A_{i^\prime j} \ne 0,$$

showing $z_i$ and $z_{i^\prime}$ are not independent, which means $\mathcal{P}(A)$ is false, QED.

$\endgroup$
3
$\begingroup$

For orthogonal columns, in general it's not. Let $$A=\begin{bmatrix}1 &-1\\1&1\end{bmatrix}$$

This results in $z_1=x_1-x_2,z_2=x_1+x_2$. These two aren't independent in general. For example, assume $x_i$ are Bernoulli RVs, then if $z_2=2$, $z_1$ is definitely $0$, which means there is dependency.

For the linear independence, see @David's counter-example.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.