# Normal distributed random variables with constraint?

Consider $$n$$ random variables $$X_i$$ with $$i=1,2,...,n$$, each drawing values from identical normal distributions with mean $$\mu=0$$ and standard deviation $$\sigma=const.$$ so that expectation values are $$\langle X_i\rangle=0$$ and $$\langle X_i^2\rangle=const.$$ Is there a way to modify these distributions to additionally enforce

$$\sum_{i=1}^nX_i=0$$

not just on average, but for each set of elements drawn? How can one generate and parameterize explicit distribution functions to numerically produce such sets of elements?

Naively, I suppose one could generate elements from the original distribution without the extra constraint and project to the subset of elements that satisfies the additional constraint... But that seems a bit unsatisfactory. Any analytical way to incorporate the constraint?

Assuming you intend for the variables $$X_1,...,X_n$$ to be independent, there is no way to preserve all the specified conditions, since independence of the variables is incompatible with the summation constraint. The closest thing you could do here is to enforce this constraint by subtracting the sample mean, which gives you the corresponding variables $$Y_i \equiv X_i - \bar{X}_n$$. These latter variables satisfy the constraint:

$$\sum_{i=1}^n Y_i = 0,$$

but they no longer follow the original joint distribution of the underlying $$X_i$$ values, and in particular, they are no longer independent variables. To see the joint distribution of the latter variables, we define the idempotent matrix $$\mathbf{T} \equiv \boldsymbol{I}_n - \mathbf{1}_{n \times n}/n$$ and we then have:$$^\dagger$$

$$\mathbf{Y} = \mathbf{T} \mathbf{X} \sim \text{N} (\mathbf{0}, \sigma^2 \mathbf{T}).$$

This means that $$\mathbb{V}(Y_i) = \tfrac{n-1}{n} \cdot \sigma^2$$ and $$\mathbb{C}(Y_i,Y_j) = -\tfrac{1}{n} \cdot \sigma^2$$ so we see that there is negative correlation between the variables. The constraint is now built into the joint distribution.

$$^\dagger$$ Note that one of the eigenvalues of the matrix $$\mathbf{T}$$ is zero, and it does not have full rank. Hence, the variance matrix in this expression is non-negative definite, but not positive definite. This gives a special case of the normal distribution called the singular normal distribution (see e.g., Kwong and Iglewicz 1996). In this special case of the multivariate normal distribution, one of the values is constrained by the others.

• Thank you, this looks very good! Especially knowing the variance and correlations is useful. I'm a bit confused with the notation though, could you perhaps clarify which vector spaces the $I_n$ and $1_{n\times n}$ act on? And shouldn't the sample mean $\bar X_n$ vanish due to zero mean of all initial distributions? – Kagaratsch Jan 31 '19 at 11:03
• Zero mean of the distribution doesn't lead to zero sample mean. Instead you have $\bar{X}_n \sim \text{N}(0, \sigma^2/n)$. – Ben Jan 31 '19 at 11:06
• I see, makes sense, it is only really zero for $n\to\infty$. – Kagaratsch Jan 31 '19 at 11:06
• Those are $n \times n$ matrices, so they operate as linear functions $\mathbb{R}^n \rightarrow \mathbb{R}^n$. – Ben Jan 31 '19 at 11:07
• I guess you mean that $I_n$ is the matrix $\delta_{i,j}$, while $1_{n\times n}$ is matrix of full dimension with all elements filled by the same number $1$. Honestly, without knowing what we want to represent $Y_i=X_i-\bar X_n=X_i-\frac{1}{n}\sum_i^nX_i$, I would not have guessed that from the notation... ^^ – Kagaratsch Jan 31 '19 at 11:17

Given that the event $$\sum_{i=1}^nX_i=0$$ is of measure zero, one needs to define a new density with respect to a new measure on the hyperplane defined by $$\sum_{i=1}^nX_i=0$$. Without the specification of these items, which is arbitrary, the question has no meaning. In other words, one can define a Normal distribution on that hyperplane in an infinite number of different ways, even though the default solution would be to choose the Lebesgue measure on that hyperplane as the reference measure and the associated Normal distribution of dimension $$n-1$$ on that hyperplane. This is equivalent to take an orthonormal basis on $$\mathbb{R}^p$$ with first basis vector $$(1,...,1)/\sqrt{n}$$ and to use a Normal distribution of dimension $$n$$ with $$p\times p$$ covariance matrix $$\Sigma=\left(\begin{matrix} 1 &1 &1 &\cdots &1\\ 0 &1 &0 &\cdots &0\\ \vdots &\vdots &\vdots &\ddots &\vdots \\ 0 &0 &0 &\cdots &1\\ \end{matrix} \right)$$