3
$\begingroup$

I don't have any idea how to go through this if anyone know how to or know what this problem is called please tell me, this trouble comes up when trying to evaluate the variance of a decision tree (not terminal nodes).

$\endgroup$
1

1 Answer 1

3
$\begingroup$

Although you don't specify, I'll assume that you have random variables $X_1,X_2,X_3,...$ that are IID. Suppose that these random variables have finite mean $\mu$ and finite variance $\sigma^2$, and suppose the random variable $N$ has finite mean $\mu_N$ and finite variance $\sigma_N^2$.

To facilitate the analysis, let $S_n = \sum_{i=1}^n x_i$ denote the sum of the first $n$ variables in the sequence. This quantity has mean and variance given respectively by:

$$\begin{align} \mathbb{E}(S_n) &= \mathbb{E} \Bigg( \sum_{i=1}^n X_i \Bigg) = \sum_{i=1}^n \mathbb{E} (X_i) = \sum_{i=1}^n \mu = n \mu, \\[12pt] \mathbb{V}(S_n) &= \mathbb{V} \Bigg( \sum_{i=1}^n X_i \Bigg) = \sum_{i=1}^n \mathbb{V} (X_i) = \sum_{i=1}^n \sigma^2 = n \sigma^2. \\[12pt] \end{align}$$

Consequently, using the law of iterated variance you get:

$$\begin{align} \mathbb{V}(S_N) &= \mathbb{E}(\mathbb{V}(S_N | N) ) + \mathbb{V}(\mathbb{E}(S_N | N) ) \\[6pt] &= \mathbb{E}( N \sigma^2 ) + \mathbb{V}( N \mu ) \\[6pt] &= \sigma^2 \mathbb{E}(N) + \mu^2 \mathbb{V}(N) \\[6pt] &= \sigma^2 \mu_N + \mu^2 \sigma_N^2. \\[6pt] \end{align}$$

So, if you can obtain the mean and variance of $X_i$ and $N$, it is simple to obtain the variance of $S_N$.

$\endgroup$
7
  • $\begingroup$ Shouldn't you know the $S_N|N$ distribution to evaluate $V(S_N|N)$? you have evaluated it like $N$ was not a r.v. For example, $Z=a.X$, $a$ constant not r.v so far, so its var is equal to $Z=a^2.V(X)$ now take $a$ to be r.v so Is $V(Z|a)=a^2.V(X)$? $\endgroup$ Oct 25, 2021 at 1:21
  • 2
    $\begingroup$ No, you don't need to know the distribution of $S_N|N$ to evaluate $\mathbb{V}(S_N|N)$. Since the $X_i$ are i.i.d., it follows that $\mathbb{V}(S_N|N) = N*\sigma^2$, regardless of distribution. And yes, you do evaluate it like $N$ is not an r.v. The outer expectation $\mathbb{E}$ handles the randomness of $N$ by taking the expectation over $N$, and similarly with respect to the second term on the r.h.s. $\endgroup$
    – jbowman
    Oct 25, 2021 at 1:42
  • $\begingroup$ Thank you I would never figure it out, In this case would $V(S_N|N) \neq V(S_N|N=n) $ right? as if you had used $V(S_N|N=n)$ this would come out from the expectation I mean $E(V(S_N|N=n))=E(n.\sigma^2)=n.\sigma^2$ $\endgroup$ Oct 25, 2021 at 2:20
  • 1
    $\begingroup$ Yes, that is right --- $\mathbb{V}(S_N|N) = N \sigma^2$ (which is a random variable) whereas $\mathbb{V}(S_N|N=n) = n \sigma^2$ (which is a constant). $\endgroup$
    – Ben
    Oct 25, 2021 at 2:40
  • 1
    $\begingroup$ Variance of Poisson distribution is $\lambda$, not $\lambda^2$. Once you correct that you should get rough correspondence of variance (subject to random error from simulations). You should have real_var = 9*5+17^2*5 which is 1490. $\endgroup$
    – Ben
    Oct 25, 2021 at 21:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.