2
$\begingroup$

If $\mathbf{X} \sim \mathcal{N}_n(\mathbf{\mu}, \mathbf{\Sigma})$, what is the joint distribution of $(\mathbf{X}, \sum_{i=1}^n c_i X_i)$ where $c_i$ are constants?

I've tried to work this out, but haven't managed it so far.

Thanks for any help,

Jack

$\endgroup$
2
$\begingroup$

For an affine transformation of a random vectors, the following rules apply. Let $\mathbf{X}$ be the $n\times 1$ column vector with random variables $X_1, X_2, \ldots, X_n$ as its elements. Let $\mathbf{a}$ and $\mathbf{b}^T$ be given matrices of size $k\times 1$ and $k\times n$ then $$ \mathbf{Y} = \mathbf{a} + \mathbf{b}^T \mathbf{X} $$ is a random vector of size $k\times 1$. Now let $\boldsymbol{\mu} = \text{E}[\mathbf{X}]$ and $\boldsymbol{\Sigma} = \text{Cov}[\mathbf{X}]$, then the corresponding mean and covariance matrix of $\mathbf{Y}$ are \begin{align*} \text{E}[\mathbf{Y}] & = \mathbf{a} + \mathbf{b}^T \text{E}[\mathbf{X}] = \mathbf{a} + \mathbf{b}^T \boldsymbol{\mu} \\ \text{Cov}[\mathbf{Y}] & = \mathbf{b}^T \text{Cov}[\mathbf{X}] \mathbf{b} = \mathbf{b}^T \boldsymbol{\Sigma} \mathbf{b} \end{align*} In your case, the transformation from $\mathbf{X}$ to the column vector $$ \mathbf{Y} = [X_1\ X_2\ \ldots\ X_n\ \sum_{j=1}^n c_jX_j]^T $$ is obtained with $$ \mathbf{a} = \mathbf{0} \qquad \mathbf{b}^T = \begin{bmatrix} \mathbf{I}_n \\ \mathbf{c}^T \end{bmatrix} $$ where $\mathbf{c}^T = [c_1\ c_2\ \ldots\ c_n]$. So that gives \begin{align*} \mathbf{Y} & = \begin{bmatrix} \mathbf{I}_n\\ \mathbf{c}^T \end{bmatrix} \mathbf{X}\\ \text{E}[\mathbf{Y}] & = \begin{bmatrix} \mathbf{I}_n\\ \mathbf{c}^T \end{bmatrix} \boldsymbol{\mu} = \begin{bmatrix} \boldsymbol{\mu} \\ \mathbf{c}^T \boldsymbol{\mu} \end{bmatrix}\\ \text{Cov}[\mathbf{Y}] & = \begin{bmatrix} \mathbf{I}_n\\ \mathbf{c}^T \end{bmatrix} \boldsymbol{\Sigma} \begin{bmatrix} \mathbf{I}_n & \mathbf{c} \end{bmatrix} = \begin{bmatrix} \boldsymbol{\Sigma} & \boldsymbol{\Sigma}\mathbf{c} \\ \mathbf{c}^T \boldsymbol{\Sigma} & \mathbf{c}^T\boldsymbol{\Sigma}\mathbf{c}\end{bmatrix} \end{align*} We can conclude that $\mathbf{Y}$ is multivariate normal with expectation and covariance matrix given above. An important issue here, as already indicated by @gunes, is that $\text{Cov}[\mathbf{Y}]$ is not positive definite, even if $\boldsymbol{\Sigma}$ is. That means the joint distribution of $\mathbf{Y}$ is "degenerate" in the sense that its values only occupy a subspace of dimension $n$ (or less if $\boldsymbol{\Sigma}$ is also not positive definite) instead of $n+1$.

$\endgroup$
0
$\begingroup$

The last variable is linearly dependent on the first $n$. Let the joint PDF of the original random vector be $f_{\mathbf{X}}(\mathbf{x})$. And, denote the new PDF as $f_{\mathbf{X}_y}(\mathbf{x},y)$. Then, since the linear relation between $x_i$ and $y$ must hold: $$f_{\mathbf{X}_y}(\mathbf{x},y)=f_\mathbf{X}(\mathbf{x})\delta\left(y-\sum_{i=1}^n x_i\right)$$

The dirac-delta is there in order to enforce the linear dependence. If the linear relation doesn't hold, the PDF is $0$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.