# Joint distribution of random vector and a linear combination of it

If $$\mathbf{X} \sim \mathcal{N}_n(\mathbf{\mu}, \mathbf{\Sigma})$$, what is the joint distribution of $$(\mathbf{X}, \sum_{i=1}^n c_i X_i)$$ where $$c_i$$ are constants?

I've tried to work this out, but haven't managed it so far.

Thanks for any help,

Jack

For an affine transformation of a random vectors, the following rules apply. Let $$\mathbf{X}$$ be the $$n\times 1$$ column vector with random variables $$X_1, X_2, \ldots, X_n$$ as its elements. Let $$\mathbf{a}$$ and $$\mathbf{b}^T$$ be given matrices of size $$k\times 1$$ and $$k\times n$$ then $$\mathbf{Y} = \mathbf{a} + \mathbf{b}^T \mathbf{X}$$ is a random vector of size $$k\times 1$$. Now let $$\boldsymbol{\mu} = \text{E}[\mathbf{X}]$$ and $$\boldsymbol{\Sigma} = \text{Cov}[\mathbf{X}]$$, then the corresponding mean and covariance matrix of $$\mathbf{Y}$$ are \begin{align*} \text{E}[\mathbf{Y}] & = \mathbf{a} + \mathbf{b}^T \text{E}[\mathbf{X}] = \mathbf{a} + \mathbf{b}^T \boldsymbol{\mu} \\ \text{Cov}[\mathbf{Y}] & = \mathbf{b}^T \text{Cov}[\mathbf{X}] \mathbf{b} = \mathbf{b}^T \boldsymbol{\Sigma} \mathbf{b} \end{align*} In your case, the transformation from $$\mathbf{X}$$ to the column vector $$\mathbf{Y} = [X_1\ X_2\ \ldots\ X_n\ \sum_{j=1}^n c_jX_j]^T$$ is obtained with $$\mathbf{a} = \mathbf{0} \qquad \mathbf{b}^T = \begin{bmatrix} \mathbf{I}_n \\ \mathbf{c}^T \end{bmatrix}$$ where $$\mathbf{c}^T = [c_1\ c_2\ \ldots\ c_n]$$. So that gives \begin{align*} \mathbf{Y} & = \begin{bmatrix} \mathbf{I}_n\\ \mathbf{c}^T \end{bmatrix} \mathbf{X}\\ \text{E}[\mathbf{Y}] & = \begin{bmatrix} \mathbf{I}_n\\ \mathbf{c}^T \end{bmatrix} \boldsymbol{\mu} = \begin{bmatrix} \boldsymbol{\mu} \\ \mathbf{c}^T \boldsymbol{\mu} \end{bmatrix}\\ \text{Cov}[\mathbf{Y}] & = \begin{bmatrix} \mathbf{I}_n\\ \mathbf{c}^T \end{bmatrix} \boldsymbol{\Sigma} \begin{bmatrix} \mathbf{I}_n & \mathbf{c} \end{bmatrix} = \begin{bmatrix} \boldsymbol{\Sigma} & \boldsymbol{\Sigma}\mathbf{c} \\ \mathbf{c}^T \boldsymbol{\Sigma} & \mathbf{c}^T\boldsymbol{\Sigma}\mathbf{c}\end{bmatrix} \end{align*} We can conclude that $$\mathbf{Y}$$ is multivariate normal with expectation and covariance matrix given above. An important issue here, as already indicated by @gunes, is that $$\text{Cov}[\mathbf{Y}]$$ is not positive definite, even if $$\boldsymbol{\Sigma}$$ is. That means the joint distribution of $$\mathbf{Y}$$ is "degenerate" in the sense that its values only occupy a subspace of dimension $$n$$ (or less if $$\boldsymbol{\Sigma}$$ is also not positive definite) instead of $$n+1$$.

The last variable is linearly dependent on the first $$n$$. Let the joint PDF of the original random vector be $$f_{\mathbf{X}}(\mathbf{x})$$. And, denote the new PDF as $$f_{\mathbf{X}_y}(\mathbf{x},y)$$. Then, since the linear relation between $$x_i$$ and $$y$$ must hold: $$f_{\mathbf{X}_y}(\mathbf{x},y)=f_\mathbf{X}(\mathbf{x})\delta\left(y-\sum_{i=1}^n x_i\right)$$

The dirac-delta is there in order to enforce the linear dependence. If the linear relation doesn't hold, the PDF is $$0$$.