Can a joint second moment exist if the joint distribution cannot be derived? I am struggling with writing down a relationship between two variables and I am happy for each and every comment that points out, how I can overcome this problem:
Let $x\sim N(\mu,\Sigma), x \in\mathbb{R}^{N-1}$ be a vector that is multivariate normally distributed with mean parameter $\mu$ and covariance matrix $\Sigma$. $x_N$ is of such that $\sum_{i=1}^{N}x_i=1$. (Therefore: conditional on $x$ the scalar $x_N$ is deterministic defined as $1-\sum x$.) It is a standard result that $X_n$ (unconditional of $x$) as the sum of normal distributed variables is also normally distributed with mean parameter $1-\iota'x$ (where $\iota$ is a vector of ones) and variance $\iota'\Sigma\iota$.
However, it is an obstacle for me to write down the joint distribution of $z:=(x,x_N)$ (which is apparently not normal due to the deterministic behavior of $x_N$ given $x$).
However, I can easily sample from the joint distribution and it is not surprising to see for me that the expected value of $z=E(z)=(\mu,1-\iota'\mu)$. Sampling also gives me that the covariance between $x_N$ and $x_i$ ($i\in\{1,\ldots,N-1\}$) is what I would expect:
$$cov(x_N,x_i)=cov(1-\sum_{j=1}^{N-1}x_j,x_i)=-\sum_{j=1}^{N-1}cov(x_j,x_i)$$ which is nothing but minus the sum of the $i$-th  row of $\Sigma$. Can anyone help me to explain how to mathematically show that this is correct although I don't have the joint distribution of $z$?
 A: Suppose that random variables $X_1,X_2,\ldots,X_{N-1}$ have a
a multivariate normal distribution (jointly normal) with mean vector
$\mathbf m = (\mu_1,\mu_2, \ldots, \mu_{N-1})$ and
covariance matrix $C$.  Define
$$X_N = 1 - \sum_{i=1}^{N-1}X_i.$$ 
Then, $X_N$ is also normally distributed with mean 
$\displaystyle E[X_N] = \mu_N = 1 - \sum_{i=1}^{N-1} \mu_i$
and variance
$$\sigma_N^2 = \operatorname{cov}(X_N,X_N) 
= \operatorname{cov}\left(\sum_{i=1}^{N-1}X_i, \sum_{j=1}^{N-1}X_j\right)
= \sum_{i=1}^{N-1}\sum_{j=1}^{N-1} C_{i,j}.$$
Furthermore, the transformation
$(X_1,X_2,\ldots, X_{N-1})\to (X_1,X_2,\ldots, X_{N-1}, X_N)$
is a linear transformation (well, actually an affine transformation
if you want to nitpick) of jointly normal random variables,
and so $(X_1,X_2,\ldots, X_{N-1}, X_N)$ also enjoys a jointly
normal distribution with mean vector 
$\mathbf m^\prime = (\mathbf m, \mu_N) = (\mu_1,\mu_2, \ldots, 
\mu_{N-1}, \mu_N)$ and covariance matrix $\Sigma$ which 
is a bordered $N\times N$ matrix in which 


*

*$C$ is the $(N-1)\times(N-1)$
submatrix in the upper left hand corner, 

*$\Sigma_{N,N} = \sigma_N^2$ appears in the lower right hand corner

*the $(N,j)$-th entry in the bottom row of $\Sigma$, which is
$\operatorname{cov}(X_N,X_j)$, has value
$$\Sigma_{N,j} = \operatorname{cov}(X_N,X_j) =
\operatorname{cov}\left(1-\sum_{i=1}^{N-1}X_i,X_j\right)
= -\left(\sum_{i=1}^{N-1}\operatorname{cov}(X_i,X_j)\right)
= -\sum_{i=1}^{N-1} C_{i,j},$$ that is, $\Sigma_{N,j}$
is the negative of the sum of the entries in the $j$-th
column of $C$. Put another way, the column sums of $\Sigma$
equal $0$, and of you think about it a little, so do the row
sums of $\Sigma$ equal $0$.
Note that all that we have used is the fact that covariance
is a bilinear function.
