# For Gaussian distribution matrix $X$, find $E(F(X))$ and $\sigma(X)$ s.t. $F=\sum f(x_i)P_i,$ given $E(f_i(x_i)),\sigma(f_i(x_i)) \forall i$

Deterministically I calculate my objective by summing situation-dependent response functions:

$$F(X)=\sum^{n}_{i=1}f_i(x_i)P_i$$ Where $x_i$ is a vector of parameters, $f_i$ is the response value (a real number) given situation $i$, and $P_i$ is the probability of situation $i$ occurring.

For uncertainty in (i.e. Gaussian probability distributions of) $x_i$ for each situation i, is it possible for me to calculate the expected objective $E(F(X))$ and the standard deviation of $F(X)$ given the mean value and standard deviation of each set of uncertain parameters $\big\{E(f_1(x_1)),E(f_2(x_2)),...,E(f_n(x_n)), \sigma(f_1(x_1)),\sigma(f_2(x_2)),...,\sigma(f_n(x_n))$ $\big\}$ and known probabilities $\big\{P_1,P_2,...,P_n \big\}$?

Edit: Here is what I've tried so far for this proof.

Our function of interest $F$ is a function of the stochastic subroutine $f$:

$F(X) = \sum_{\forall{i}}f(x_i)P_i$

where $x_i$ represents the $i^{th}$ scenario (e.g. directions considered or modes of operation).

The expected value of f can be calculated as follows:

$E(f(x))=\frac{1}{m}\sum_{j=1}^nf(x)_j$

where $\{x_j\forall j\}$ is the set of m possible realizations of the stochastic scenario $x$ and $f(x)_j$ is the $j^{th}$ possible realization of the stochastic function $f(x)$

Similarly, the expected value of F(X) can be calculated by summing all $m$ possible scenarios:

$E(F(X))=\frac{1}{m}\sum_{j=1}^m \sum_{\forall{i}}f(x_i)_jP_i$

Because I can add things in any order, the above result can be rewritten in terms of $E(f(x))$:

$E(F(X))= \sum_{\forall{i}}E(f(x_i))P_i$

I am struggling to find a simmillar result for $\sigma(F(X))$. I tried writing what I know about $\sigma(F(X))$ and $\sigma(f(X))$ but I can't connect these ideas like I did with $E(F(X))$ and $E(f(X))$.

$\sigma(f(X)) = \frac{1}{m-1}\sum_{j=1}^m(E(f(x))-f(x)_j)^2$

$=\frac{1}{m}\sum_{j=1}^m(\frac{1}{m}\sum_{k=1}^m(f(x)_k) - f(x)_j)$

$\sigma(F(X)) = \frac{1}{m}\sum_{j=1}^m(E(F(x))-F(x)_j)^2$

$= \frac{1}{m}\sum_{j=1}^m(\sum_{k=1}^m \sum_{i=1}^n f(x_i)_k - \sum_{i=1}^n f(x_i)_j)^2$

• It looks to me like $F(X)$ is already an expected value; $E(F(X))$ will just equal $F(X)$ because there's no randomness left after the summation. The variance would then just equal $\sum(f_i(x_i)-F(X))^2P_i$. Or am I missing something? – jbowman Nov 10 '16 at 1:25
• @jbowman I updated my question to more explicitly state X is now a Guassian probability function – kilojoules Nov 10 '16 at 15:52
• en.wikipedia.org/wiki/Mixture_distribution#Moments – Sextus Empiricus Jun 20 '18 at 11:25

Yes, you can calculate the expected value and standard deviation of $F(X)$.

For the expected value, $\mathbb{E}F(X) = \sum_{i=1}^n \mathbb{E}f_i(x_i)P_i$ - just the probability-weighted average of the individual expected values.

For the variance, it's a little more complicated:

$\sigma^2(F(X)) = \sum_{i=1}^n \sigma^2(f_i(x_i))P_i + \sum_{i=1}^n (\mathbb{E}f_i(x_i)-\mathbb{E}F(X))^2 P_i$

... the probability-weighted average of the individual variances plus the variance of the expected values. Then just take the square root of $\sigma^2(F(X))$ and you're done.

Edit in response to question below:

To see that this is so, consider the following derivation of a simpler case. Let $x_i$ have mean $\mu_i$ and variance $\sigma^2_i$, and the $\mu_i$ have mean and variance $\mu$ and $\sigma^2_{\mu}$ respectively. Then

$\mathbb{E}(x_i-\mu_{\mu})^2 = \mathbb{E}(x_i-\mu_i+\mu_i-\mu)^2$

$= \mathbb{E}(x_i-\mu_i)^2 + \mathbb{E}(\mu_i-\mu_{\mu})^2 + 2\mathbb{E}(x_i-\mu_i)(\mu_i-\mu_{\mu})$

The key step: as $\mathbb{E}(x_i-\mu_i) = 0$ and $(x_i-\mu_i)$ is uncorrelated with $(\mu_i - \mu_{\mu})$, the last term disappears and we have:

$\mathbb{E}(x_i-\mu_{\mu})^2 = \sigma^2_i + \sigma^2_{\mu}$

Now your case is a more complex version of this, but the key step is the same: we can add the expected variance to the variance of the expected values because the cross-product term in the expansion of the square has expected value equal to zero, so disappears.

Here we go. First, some preliminaries:

$\sigma^2(F(X)) = \mathbb{E}(f_i(x_i)-\mathbb{E}F(X))^2$

$= \mathbb{E}\sum_{i=1}^n(f_i(x_i)-\mathbb{E}F(X))^2P_i$

$= \sum_{i=1}^n\mathbb{E}(f_i(x_i)-\mathbb{E}F(X))^2P_i$

Now to expand the square:

$= \sum_{i=1}^n\mathbb{E}(f_i(x_i)-\mathbb{E}f_i(x_i) + \mathbb{E}f_i(x_i) -\mathbb{E}F(X))^2P_i$

$= \sum_{i=1}^n\mathbb{E}(f_i(x_i)-\mathbb{E}f_i(x_i))^2P_i + \sum_{i=1}^n(\mathbb{E}f_i(x_i) -\mathbb{E}F(X))^2P_i + 2\sum_{i=1}^n\mathbb{E}(f_i(x_i) -\mathbb{E}f_i(x_i))(\mathbb{E}f_i(x_i) -\mathbb{E}F(X))P_i$

The last term evaluates to 0, as $\mathbb{E}(f_i(x_i) -\mathbb{E}f_i(x_i)) = \mathbb{E}f_i(x_i) -\mathbb{E}f_i(x_i) = 0$. We drop it and rewrite:

$\sigma^2(F(X)) = \sum_{i=1}^n\mathbb{E}(f_i(x_i)-\mathbb{E}f_i(x_i))^2P_i + \sum_{i=1}^n(\mathbb{E}f_i(x_i) -\mathbb{E}F(X))^2P_i$

$= \sum_{i=1}^n \sigma^2(f_i(x_i))P_i + \sum_{i=1}^n (\mathbb{E}f_i(x_i)-\mathbb{E}F(X))^2 P_i$

• The means inside your summation are conditional on the "situation" (event) among the $n$ possible ones, right? – DeltaIV Nov 11 '16 at 9:45
• @jbowman Can you explain why the second part is correct? I am having trouble following your reasoning. – kilojoules Nov 23 '16 at 22:30
• How can we know we can add the $f(x_i)$ variances with the variance of the expected values? I'm not sure why that would be true. – kilojoules Nov 24 '16 at 18:43
• I've expanded my answer to answer your followup question as well. – jbowman Nov 24 '16 at 21:46
• @jbowman I was not able to use your clue to complete this proof. I edited my question to show what I have tried so far and where I'm stuck, – kilojoules Nov 28 '16 at 18:32