If $X_1,\cdots,X_n \sim \mathcal{N}(\mu, 1)$ are IID, then compute $\mathbb{E}\left( X_1 \mid T \right)$, where $T = \sum_i X_i$

Question

Question

If $X_1,\cdots,X_n \sim \mathcal{N}(\mu, 1)$ are IID, then compute $\mathbb{E}\left( X_1 \mid T \right)$, where $T = \sum_i X_i$.

---

Attempt: Please check if the below is correct.

Let say, we take the sum of the those conditional expectations such that, \begin{align} \sum_i \mathbb{E}\left( X_i \mid T \right) = \mathbb{E}\left( \sum_i X_i \mid T \right) = T . \end{align} It means that each $\mathbb{E}\left( X_i \mid T \right) = \frac{T}{n}$ since $X_1,\ldots,X_n$ are IID.

Thus, $\mathbb{E}\left( X_1 \mid T \right) = \frac{T}{n}$. Is it correct?

Answer 1

The idea's right--but there's a question of expressing it a little more rigorously. I will therefore focus on notation and on exposing the essence of the idea.

---

Let's begin with the idea of exchangeability:

A random variable $\mathbf X=(X_1, X_2, \ldots, X_n)$ is exchangeable when the distributions of the permuted variables $\mathbf{X}^\sigma=(X_{\sigma(1)}, X_{\sigma(2)}, \ldots, X_{\sigma(n)})$ are all the same for every possible permutation $\sigma$.

Clearly iid implies exchangeable.

As a matter of notation, write $X^\sigma_i = X_{\sigma(i)}$ for the $i^\text{th}$ component of $\mathbf{X}^\sigma$ and let $$T^\sigma = \sum_{i=1}^n X^\sigma_i = \sum_{i=1}^n X_i = T.$$

Let $j$ be any index and let $\sigma$ be any permutation of the indices that sends $1$ to $j = \sigma(1).$ (Such a $\sigma$ exists because one can always just swap $1$ and $j.$) Exchangeability of $\mathbf X$ implies

$$E[X_1\mid T] = E[X^\sigma_1\mid T^\sigma] = E[X_j\mid T],$$

because (in the first inequality) we have merely replaced $\mathbf X$ by the identically distributed vector $\mathbf X^\sigma.$ This is the crux of the matter.

Consequently

$$T = E[T \mid T] = E[\sum_{i=1}^n X_i\mid T] = \sum_{i=1}^n E[X_i\mid T] = \sum_{i=1}^n E[X_1\mid T] = n E[X_1 \mid T],$$

whence

$$E[X_1\mid T] = \frac{1}{n} T.$$

Answer 2

$\newcommand{\one}{\mathbf 1}$This is not a proof (and +1 to @whuber's answer), but it's a geometric way to build some intuition as to why $E(X_1 | T) = T/n$ is a sensible answer.

Let $X = (X_1,\dots,X_n)^T$ and $\one = (1,\dots,1)^T$ so $T = \one^TX$. We're then conditioning on the event that $\one^TX = t$ for some $t \in \mathbb R$, so this is like drawing multivariate Gaussians supported on $\mathbb R^n$ but only looking at the ones that end up in the affine space $\{x \in \mathbb R^n : \one^Tx = t\}$. Then we want to know the average of the $x_1$ coordinates of the points that land in this affine space (never mind that it's a measure zero subset).

We know $$ X \sim \mathcal N(\mu \one, I) $$ so we've got a spherical Gaussian with a constant mean vector, and the mean vector $\mu\one$ is on the same line as the normal vector of the hyperplane $x^T\one = 0$.

This gives us a situation like the picture below:

The key idea: first imagine the density over the affine subspace $H_t := \{x : x^T\one = t\}$. The density of $X$ is symmetric around $x_1 = x_2$ since $E(X) \in \text{span } \one$. The density will also be symmetric on $H_t$ as $H_t$ is also symmetric over the same line, and the point around which it is symmetric is the intersection of the lines $x_1 + x_2 = t$ and $x_1 = x_2$. This happens for $x = (t/2, t/2)$.

To picture $E(X_1 | T)$ we can imagine sampling over and over, and then whenever we get a point in $H_t$ we take just the $x_1$ coordinate and save that. From the symmetry of the density on $H_t$ the distribution of the $x_1$ coordinates will also be symmetric, and it'll have the same center point of $t/2$. The mean of a symmetric distribution is the central point of symmetry so this means $E(X_1 | T) = T/2$, and that $E(X_1| T) = E(X_2 | T)$ since $X_1$ and $X_2$ can be excahnged without affecting anything.

In higher dimensions this gets hard (or impossible) to exactly visualize, but the same idea applies: we've got a spherical Gaussian with a mean in the span of $\one$, and we're looking at an affine subspace that's perpendicular to that. The balance point of the distribution on the subspace will still be the intersection of $\text{span }\one$ and $\{x : x^T\one = t\}$ which is at $x=(t/n, \dots, t/n)$, and the density is still symmetric so this balance point is again the mean.

Again, that's not a proof, but I think it gives a decent idea of why you'd expect this behavior in the first place.

---

Beyond this, as some such as @StubbornAtom have noted, this doesn't actually require $X$ to be Gaussian. In 2-D, note that if $X$ is exchangeable then $f(x_1, x_2) = f(x_2, x_1)$ (more generally, $f(x) = f(x^\sigma)$) so $f$ must be symmetric over the line $x_1 = x_2$. We also have $E(X) \in \text{span }\one$ so everything I said regarding the "key idea" in the first picture still exactly holds. Here's an example where the $X_i$ are iid from a Gaussian mixture model. All the lines have the same meaning as before.

Answer 3

I think your answer is right, although I'm not entirely sure about the killer line in your proof, about it being true "because they are i.i.d". A more wordy way to the same solution is as follows:

Think about what $\mathbb{E}(x_{i}|T)$ actually means. You know that you have a sample with N readings and that their mean is T. What this actually means, is that now, the underlying distribution they were sampled from no longer matters (you'll notice you at no point used the fact it was sampled from a Gaussian in your proof).

$\mathbb{E}(x_{i}|T)$ is the answer to the question, if you sampled from your sample, with replacement many times, what would be the average you obtained. This is the sum over all the possible values, multiplied by their probability, or $\sum_{i=1}^{N}\frac{1}{N}x_{i}$ which equals T.

Answer 4

Proof:

Let $u_1\times u_2 \times...\times u_n$ be the product measure $\mathbb{P}=u_1\times u_2 \times...\times u_n$ of these probability space where the random vector ($X_1,X_2,...,X_n$) lives.

Let $F(x_1, x_2, x_3, ... , x_n)$ be the joint distribution function of random vector $X_1,X_2,...,X_n$

Easy to know that $(u_1 \circ X_1^{-1}) \times (u_2 \circ X_2^{-1}) \times...\times (u_n \circ X_n^{-1}) = F$

Since $X_i$ is i.i.d., for each permutation $p$ of $p$($X_1,X_2,...,X_n$)=($X_{p 1},X_{p 2},...,X_{p n}$), we have $F(x_1, x_2, ... , x_n) = F(x_{p 1}, x_{p 2}, ... , x_{p n})$

Then according to the definition of conditional expectation:

For each $A \in \sigma(T)$ , we have $A=\left\{X_1+X_2+..+X_n \in C \right\}$ for $C \in \mathcal{B}$

$\int_{A}{\mathbb{E}(X_1|T)d\mathbb{P}}=\int_{A}{X_1d\mathbb{P}}$

$=\int_{A}{X_1d(\mathbb{u_1\times u_2 \times...\times u_n})}$

$=\int_{C}{x_1F(dx_1,dx_2,...dx_n)}$

Change variables : $x_{p i} = x_{i}$, for $i = 1,2,...,n$

$= \int_{C}{x_{\sigma 1}F(dx_{p 1}, dx_{p 2}, ..., dx_{p n})}$

$= \int_{C}{x_{\sigma 1}F(dx_{1}, dx_{2}, ..., dx_{n})}$

$= \int_{A}{X_{\sigma 1}d(\mathbb{u_1\times u_2 \times...\times u_n})}$

$= \int_{A}{X_{\sigma 1}d\mathbb{P}}$

$= \int_{A}{\mathbb{E}(X_{p 1}|T)d\mathbb{P}}$

$\Rightarrow \mathbb{E}(X_{p 1}|T) = \mathbb{E}(X_1|T)$ a.s. for each permutation $p$

$\Rightarrow \mathbb{E}(X_{1}|T) = \mathbb{E}(T|T)/n = T/n$

Q.E.D