I don't have any idea how to go through this if anyone know how to or know what this problem is called please tell me, this trouble comes up when trying to evaluate the variance of a decision tree (not terminal nodes).
$\begingroup$
$\endgroup$
1
-
$\begingroup$ Maybe try the total expectation theorem, see stats.stackexchange.com/questions/431667/… for an example $\endgroup$– kjetil b halvorsen ♦Commented Oct 25, 2021 at 0:36
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
7
Although you don't specify, I'll assume that you have random variables $X_1,X_2,X_3,...$ that are IID. Suppose that these random variables have finite mean $\mu$ and finite variance $\sigma^2$, and suppose the random variable $N$ has finite mean $\mu_N$ and finite variance $\sigma_N^2$.
To facilitate the analysis, let $S_n = \sum_{i=1}^n x_i$ denote the sum of the first $n$ variables in the sequence. This quantity has mean and variance given respectively by:
$$\begin{align} \mathbb{E}(S_n) &= \mathbb{E} \Bigg( \sum_{i=1}^n X_i \Bigg) = \sum_{i=1}^n \mathbb{E} (X_i) = \sum_{i=1}^n \mu = n \mu, \\[12pt] \mathbb{V}(S_n) &= \mathbb{V} \Bigg( \sum_{i=1}^n X_i \Bigg) = \sum_{i=1}^n \mathbb{V} (X_i) = \sum_{i=1}^n \sigma^2 = n \sigma^2. \\[12pt] \end{align}$$
Consequently, using the law of iterated variance you get:
$$\begin{align} \mathbb{V}(S_N) &= \mathbb{E}(\mathbb{V}(S_N | N) ) + \mathbb{V}(\mathbb{E}(S_N | N) ) \\[6pt] &= \mathbb{E}( N \sigma^2 ) + \mathbb{V}( N \mu ) \\[6pt] &= \sigma^2 \mathbb{E}(N) + \mu^2 \mathbb{V}(N) \\[6pt] &= \sigma^2 \mu_N + \mu^2 \sigma_N^2. \\[6pt] \end{align}$$
So, if you can obtain the mean and variance of $X_i$ and $N$, it is simple to obtain the variance of $S_N$.
-
$\begingroup$ Shouldn't you know the $S_N|N$ distribution to evaluate $V(S_N|N)$? you have evaluated it like $N$ was not a r.v. For example, $Z=a.X$, $a$ constant not r.v so far, so its var is equal to $Z=a^2.V(X)$ now take $a$ to be r.v so Is $V(Z|a)=a^2.V(X)$? $\endgroup$ Commented Oct 25, 2021 at 1:21
-
2$\begingroup$ No, you don't need to know the distribution of $S_N|N$ to evaluate $\mathbb{V}(S_N|N)$. Since the $X_i$ are i.i.d., it follows that $\mathbb{V}(S_N|N) = N*\sigma^2$, regardless of distribution. And yes, you do evaluate it like $N$ is not an r.v. The outer expectation $\mathbb{E}$ handles the randomness of $N$ by taking the expectation over $N$, and similarly with respect to the second term on the r.h.s. $\endgroup$– jbowmanCommented Oct 25, 2021 at 1:42
-
$\begingroup$ Thank you I would never figure it out, In this case would $V(S_N|N) \neq V(S_N|N=n) $ right? as if you had used $V(S_N|N=n)$ this would come out from the expectation I mean $E(V(S_N|N=n))=E(n.\sigma^2)=n.\sigma^2$ $\endgroup$ Commented Oct 25, 2021 at 2:20
-
1$\begingroup$ Yes, that is right --- $\mathbb{V}(S_N|N) = N \sigma^2$ (which is a random variable) whereas $\mathbb{V}(S_N|N=n) = n \sigma^2$ (which is a constant). $\endgroup$– BenCommented Oct 25, 2021 at 2:40
-
1$\begingroup$ Variance of Poisson distribution is $\lambda$, not $\lambda^2$. Once you correct that you should get rough correspondence of variance (subject to random error from simulations). You should have
real_var = 9*5+17^2*5which is1490. $\endgroup$– BenCommented Oct 25, 2021 at 21:27