0
$\begingroup$

I would be interested in finding the value of the following expression:

$$\mathbb{E}[X_k^2\mid S_N]$$

where $X_k$ are iid random variables with $\mathbb{E}[X_k]=\mu$ and $\operatorname{Var}[X_k]=\sigma^2$, $N$ is a deterministic constant and:

$$S_N=\sum_{i=1}^N X_i$$

I tried to find it by using that

$$\mathbb{E}[S_N^2]=N\mathbb{E}[X_k^2 \mid S_N]+N(N-1)\mathbb{E}[X_i X_j\mid S_N]$$

with $i\neq j$. Nevertheless, even if the different $X_k$ are independent, they do not necessarily satisfy $\mathbb{E}[X_i X_j\mid S_N]=\mathbb{E}[X_i\mid S_N]^2$

Edit: Another approach I tried was using the fact that

$$\mathbb{E}[X_k^2\mid S_N] = \text{Var}(X_k\mid S_N) - \mathbb{E}[X_k\mid S_N]^2$$

The second term of the sum can easily be shown to be $S_N^2/N^2$. Nevertheless, the conditional variance is still not easy for me to crack. I tried relating $\text{Var}(X_k\mid S_N)$ to $\text{Cov}(X_k, S_N) = \sigma^2$ with no success.

$\endgroup$
9
  • $\begingroup$ You might want to start by looking at specific distributions. For example, $\mathbb{E}[X_k^2|S_N]=\frac{(N-1) N \sigma ^2+S_N^2}{N^2}$ when $X_k \sim N(\mu, \sigma^2)$. $\endgroup$
    – JimB
    Commented Jul 19 at 16:19
  • $\begingroup$ Your equation $\mathbb{E}[S_N^2]=N\mathbb{E}[X_k^2 \mid S_N]+N(N-1)\mathbb{E}[X_i X_j\mid S_N]$ has problems. There is no justification or rationale for including $\mid S_N$ on the righthand side. The equation is essentially correct if you remove that resulting in $\mathbb{E}[S_N^2]=N\mathbb{E}[X_k^2]+N(N-1)\mathbb{E}[X_i X_j]$. Again, consider a specific distribution. For a normal distribution $X_1$ and $S_N$ are distributed as a bivariate normal. $\endgroup$
    – JimB
    Commented Jul 20 at 16:04
  • $\begingroup$ the answer is $\sigma^2+\mu^2$, because $S_N$ has no impact on $X_k^2$ $\endgroup$
    – Aksakal
    Commented Jul 21 at 1:44
  • 1
    $\begingroup$ @Aksakal $X_k$ and $S_N=\sum_{i=1}^N X_i$ (with $1\leq k \leq N$) are not independent so the expectation of $X_k^2$ given $S_N=s$ is not $\sigma^2+\mu^2$ (unless $s=\pm\sqrt{\mu ^2 n^2+n\sigma ^2}$ when $X_k\sim N(\mu, \sigma^2)$). $\endgroup$
    – JimB
    Commented Jul 21 at 3:55
  • 1
    $\begingroup$ @JimB, I dont think so, because that link talks about the joint probability of variables. OP is asking about marginals $\endgroup$
    – Aksakal
    Commented Jul 22 at 14:21

1 Answer 1

3
$\begingroup$

I suspect the only symbolic results you'll be able to find involve those distributions whose sum of iid random variables follow the same family of distributions. That is certainly the case for normal and gamma distributions.

Rather than show a step-by-step calculus approach, I'm going to just give handwaving arguments and Mathematica commands and I've slightly differed from your notation. You can fill in details if desired.

Normal distributions

If $X_i \sim N(\mu,\sigma^2)$, then $Y=\sum_{i=2}^n X_k$ has a normal distribution with mean $(n-1)\mu$ and variance $(n-1)\sigma^2$. $X_1$ and $S_n=X_1+Y$ have a bivariate normal distribution with mean vector $(\mu, n\mu)$ and covariance matrix

$$\Sigma=\left( \begin{array}{cc} \sigma ^2 & \sigma ^2 \\ \sigma ^2 & n \sigma ^2 \\ \end{array} \right)$$

The conditional density of $X_1$ given $S_n=s$ is found by dividing the joint density of $(X_1, S_n))$ by the marginal density of $S_n$. One then integrates $x_1^2$ times the conditional density over values of $x_1$.

Joint distribution of $X_1$ and $S_n$.

dist = TransformedDistribution[{x1, x1 + x2n},
   {x1 \[Distributed] NormalDistribution[μ, σ],
    x2n \[Distributed] NormalDistribution[(n - 1) μ, σ Sqrt[n - 1]]}];

(* PDF's of joint and marginal distributions *)
pdfx1S = PDF[dist, {x1, s}]

$$\frac{\exp \left(\frac{1}{2} \left(-\frac{(\text{x1}-\mu ) (n \text{x1}-s)}{(n-1) \sigma ^2}-\frac{(-\mu -\mu (n-1)+s) (\mu -\mu n+s-\text{x1})}{(n-1) \sigma ^2}\right)\right)}{2 \pi \sqrt{n \sigma ^4-\sigma ^4}}$$

pdfs = Simplify[PDF[MarginalDistribution[dist, 2], s], Assumptions -> s > 0]

$$\frac{e^{-\frac{(s-\mu n)^2}{2 n \sigma ^2}}}{\sqrt{2 \pi } \sqrt{n \sigma ^2}}$$

(* Find conditional expectation by integrating *)
Integrate[x1^2  pdfx1S/pdfs, {x1, -∞, ∞}, Assumptions -> {σ > 0, n > 1}]

$$\frac{(n-1) n \sigma ^2+s^2}{n^2}$$

Gamma distributions

If $X_i \sim \Gamma(k, \theta)$, then $Y=\sum_{i=2}^n X_i$ has a gamma distribution with parameters $(n-1)k$ and $\theta$. We follow the same approach as above.

Joint distribution of $X_1$ and $S_n=X_1+Y$.

dist = TransformedDistribution[{x1, x1 + x2n},
   {x1 \[Distributed] GammaDistribution[k, θ],
    x2n \[Distributed] GammaDistribution[(n - 1) k, θ]}];

(* PDF's of joint and marginal distributions *)
pdfx1S = Simplify[PDF[dist, {x1, s}] // PiecewiseExpand, Assumptions -> {s > x1 > 0}]

$$\frac{\text{x1}^{k-1} e^{-\frac{s}{\theta }} \theta ^{-k n} (s-\text{x1})^{k (n-1)-1}}{\Gamma (k) \Gamma (k (n-1))}$$

pdfs = Simplify[PDF[MarginalDistribution[dist, 2], s], Assumptions -> s > 0]

$$\frac{e^{-\frac{s}{\theta }} \theta ^{-k n} s^{k n-1}}{\Gamma (k n)}$$

(* Find conditional expectation by integrating *)
Integrate[x1^2  pdfx1S/pdfs, {x1, 0, s}, Assumptions -> {s > 0, k > 0, n > 1}]

$$\frac{(k+1) s^2}{n (k n+1)}$$

Conclusion

In short, the condition expectation depends on the common distribution and it is unlikely to obtain a symbolic solution for very many distribution families. It might seem odd that for the normal the conditional expectation is not a function of $\mu$ and for the gamma distribution the conditional expectation is not a function of $\theta$.

$\endgroup$
1
  • 2
    $\begingroup$ Similar approaches work for Possion, Binomial, and Negative binomial distributions. I'll add those in the next few days (unless someone else wants to do that). $\endgroup$
    – JimB
    Commented Jul 23 at 4:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.