Consider the $N$ i.i.d. values

$$ X_i \sim \mathcal{N}(0, \sigma^2) $$

such that

$$ Z_i = \sum_{j=1}^i X_j $$

I am interested in the distribution

$$ f(X_i | Z_N = z) $$


Under the condition, the $X_i$ are no longer independent, but they should remain identically distributed. Hence

$$ Z_N = \sum_{i=1}^N X_i = z $$


$$ \sum_{i=1}^N \mathbb{E}[X_i] = z $$

and therefore

$$ \mathbb{E}[X_i] = z/N $$


Empirically I found the variance to be

$$ var(X_i) = \frac{N-1}{N} \sigma^2 $$


If you start doing the calculations to find the distribution working backwards from $X_N$ I believe you are simply finding the product of many Gaussian distributions so I believe the distribution is going to be Gaussian. However I ran into issues with this after the first step.


How do I show rigorously that the marginal distribution

$$ (X_i | Z_N = z) \sim \mathcal{N} \left (\frac{z}{N}, \frac{N-1}{N} \sigma^2 \right) $$


2 Answers 2


It all follows from the properties of multivariate normals. Since $X_i$ are independent and normally distributed, they're jointly normal, which means any linear combination of them is also jointly normal with them. So, $p_{\mathbf{X},Z_N}(\mathbf{x},z)$ is a multivariate normal, which in turn means $p_{X_i,Z_N}(x,z)$ is multivariate normal with


Because $p_{X_i,Z_N}(x,z)$ is MV normal, the conditional distributon $p_{X_i|Z_N}(x,z)$ is univariate normal, and conditional expectation and the variance can be found (same as your answer) via the formulas under "Conditional distributions" section in the wiki page linked above.


It's a bit messy, which might explain why it isn't seen more. Here is a sketch starting from the bivariate case, which generalizes. I'll use $X$ and $Y$ and $Z=X+Y.$

First let's find the conditional cdf for $Z$ given $X=x.$

$$F_{Z|X=x}=P \left[ X+Y \leq z \ | \ X=x\right]=P[Y \leq z-x]=F_Y(z-x)$$

Then the conditional pdf is found by differentiating: $$f_{Z|X=x}=f_{Z|X}(z|x)=f_Y(z-x)$$

That in turn means the joint density function is

$$f_{X,Z}(x,z) = f_{Z|X}(z|x)f_X(x)=f_Y(z-x)f_X(x)$$

Finally, we get the conditional density formula

$$ f_{X|X+Y=z}(x,z)=\frac{f_Y(z-x)f_X(x)}{f_Z(z)} \ \ \ \ \ \ [1]$$

Now let's look at the normal case. Let $$Z=X_1 + X_2 + \cdots + X_N$$ and $$Y = X_1 + X_2 + \cdots + X_{N-1},$$ where

$$ X_i \sim \mathcal{N} \left( \mu, \sigma^2 \right) $$ and $$ Y \sim \mathcal{N} \left( \left( N - 1 \right) \mu, \left( N-1 \right) \sigma^2 \right) $$

Now what is the pdf $f_{X_N|Z}(x,z)$?

Using $[1],$

$$f_{X|Z=z}(x,z) = \frac{ \left( \frac{1} {\sqrt{{2 \pi \left(N-1 \right)\sigma^2} } } \right) \left( e^{\frac{- \left( z-x- \left( N-1 \right) \mu \right)^2 } {2 \left( N - 1 \right) \sigma^2}} \right) \left( \frac{1}{\sqrt{2 \pi \sigma^2}} \right) \left( e^{\frac{- \left( x- \mu \right)^2}{2 \sigma^2}} \right) } {\left( \frac{1} {\sqrt{{2 \pi N \sigma^2} } } \right) \left( e^{\frac{- \left( z - N \mu \right)^2}{2 N \sigma^2}} \right)}$$

After much simplifying (probably worth going throug once, but I'm not putting the details in here at this time), this can be expressed as

$$f_{X|Z=z}(x,z) =\left( \frac{1}{\sqrt{2 \pi s^2 }} \right) \left( e^{\frac{- \left( x - \frac{z}{N} \right)^2}{2 s^2}}\right), $$

where $$s^2 = \frac{\left( N - 1 \right) \sigma^2}{N}$$

This is now recognizable as a normal pdf and we can see that $$X_i|Z \sim \mathcal{N} \left( {\frac{z}{N}}, \frac{N-1}{N} \sigma^2 \right) $$ as you found empirically.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.