Deterministically I calculate my objective by summing situation-dependent response functions:

$$ F(X)=\sum^{n}_{i=1}f_i(x_i)P_i $$ Where $x_i$ is a vector of parameters, $f_i$ is the response value (a real number) given situation $i$, and $P_i$ is the probability of situation $i$ occurring.

For uncertainty in (i.e. Gaussian probability distributions of) $x_i$ for each situation i, is it possible for me to calculate the expected objective $E(F(X))$ and the standard deviation of $F(X)$ given the mean value and standard deviation of each set of uncertain parameters $\big\{E(f_1(x_1)),E(f_2(x_2)),...,E(f_n(x_n)), \sigma(f_1(x_1)),\sigma(f_2(x_2)),...,\sigma(f_n(x_n)) $ $\big\}$ and known probabilities $\big\{P_1,P_2,...,P_n \big\}$?

Edit: Here is what I've tried so far for this proof.

Our function of interest $F$ is a function of the stochastic subroutine $f$:

$F(X) = \sum_{\forall{i}}f(x_i)P_i$

where $x_i$ represents the $i^{th}$ scenario (e.g. directions considered or modes of operation).

The expected value of f can be calculated as follows:


where $\{x_j\forall j\}$ is the set of m possible realizations of the stochastic scenario $x$ and $f(x)_j$ is the $j^{th}$ possible realization of the stochastic function $f(x)$

Similarly, the expected value of F(X) can be calculated by summing all $m$ possible scenarios:

$E(F(X))=\frac{1}{m}\sum_{j=1}^m \sum_{\forall{i}}f(x_i)_jP_i$

Because I can add things in any order, the above result can be rewritten in terms of $E(f(x))$:

$E(F(X))= \sum_{\forall{i}}E(f(x_i))P_i$

I am struggling to find a simmillar result for $\sigma(F(X))$. I tried writing what I know about $\sigma(F(X))$ and $\sigma(f(X))$ but I can't connect these ideas like I did with $E(F(X))$ and $E(f(X))$.

$\sigma(f(X)) = \frac{1}{m-1}\sum_{j=1}^m(E(f(x))-f(x)_j)^2 $

$=\frac{1}{m}\sum_{j=1}^m(\frac{1}{m}\sum_{k=1}^m(f(x)_k) - f(x)_j)$

$\sigma(F(X)) = \frac{1}{m}\sum_{j=1}^m(E(F(x))-F(x)_j)^2 $

$= \frac{1}{m}\sum_{j=1}^m(\sum_{k=1}^m \sum_{i=1}^n f(x_i)_k - \sum_{i=1}^n f(x_i)_j)^2$

  • 1
    $\begingroup$ It looks to me like $F(X)$ is already an expected value; $E(F(X))$ will just equal $F(X)$ because there's no randomness left after the summation. The variance would then just equal $\sum(f_i(x_i)-F(X))^2P_i$. Or am I missing something? $\endgroup$ – jbowman Nov 10 '16 at 1:25
  • $\begingroup$ @jbowman I updated my question to more explicitly state X is now a Guassian probability function $\endgroup$ – kilojoules Nov 10 '16 at 15:52
  • $\begingroup$ en.wikipedia.org/wiki/Mixture_distribution#Moments $\endgroup$ – Sextus Empiricus Jun 20 '18 at 11:25

Yes, you can calculate the expected value and standard deviation of $F(X)$.

For the expected value, $\mathbb{E}F(X) = \sum_{i=1}^n \mathbb{E}f_i(x_i)P_i$ - just the probability-weighted average of the individual expected values.

For the variance, it's a little more complicated:

$\sigma^2(F(X)) = \sum_{i=1}^n \sigma^2(f_i(x_i))P_i + \sum_{i=1}^n (\mathbb{E}f_i(x_i)-\mathbb{E}F(X))^2 P_i$

... the probability-weighted average of the individual variances plus the variance of the expected values. Then just take the square root of $\sigma^2(F(X))$ and you're done.

Edit in response to question below:

To see that this is so, consider the following derivation of a simpler case. Let $x_i$ have mean $\mu_i$ and variance $\sigma^2_i$, and the $\mu_i$ have mean and variance $\mu$ and $\sigma^2_{\mu}$ respectively. Then

$\mathbb{E}(x_i-\mu_{\mu})^2 = \mathbb{E}(x_i-\mu_i+\mu_i-\mu)^2$

$= \mathbb{E}(x_i-\mu_i)^2 + \mathbb{E}(\mu_i-\mu_{\mu})^2 + 2\mathbb{E}(x_i-\mu_i)(\mu_i-\mu_{\mu})$

The key step: as $\mathbb{E}(x_i-\mu_i) = 0$ and $(x_i-\mu_i)$ is uncorrelated with $(\mu_i - \mu_{\mu})$, the last term disappears and we have:

$\mathbb{E}(x_i-\mu_{\mu})^2 = \sigma^2_i + \sigma^2_{\mu}$

Now your case is a more complex version of this, but the key step is the same: we can add the expected variance to the variance of the expected values because the cross-product term in the expansion of the square has expected value equal to zero, so disappears.

Here we go. First, some preliminaries:

$\sigma^2(F(X)) = \mathbb{E}(f_i(x_i)-\mathbb{E}F(X))^2$

$= \mathbb{E}\sum_{i=1}^n(f_i(x_i)-\mathbb{E}F(X))^2P_i$

$= \sum_{i=1}^n\mathbb{E}(f_i(x_i)-\mathbb{E}F(X))^2P_i$

Now to expand the square:

$ = \sum_{i=1}^n\mathbb{E}(f_i(x_i)-\mathbb{E}f_i(x_i) + \mathbb{E}f_i(x_i) -\mathbb{E}F(X))^2P_i$

$ = \sum_{i=1}^n\mathbb{E}(f_i(x_i)-\mathbb{E}f_i(x_i))^2P_i + \sum_{i=1}^n(\mathbb{E}f_i(x_i) -\mathbb{E}F(X))^2P_i + 2\sum_{i=1}^n\mathbb{E}(f_i(x_i) -\mathbb{E}f_i(x_i))(\mathbb{E}f_i(x_i) -\mathbb{E}F(X))P_i$

The last term evaluates to 0, as $\mathbb{E}(f_i(x_i) -\mathbb{E}f_i(x_i)) = \mathbb{E}f_i(x_i) -\mathbb{E}f_i(x_i) = 0$. We drop it and rewrite:

$\sigma^2(F(X)) = \sum_{i=1}^n\mathbb{E}(f_i(x_i)-\mathbb{E}f_i(x_i))^2P_i + \sum_{i=1}^n(\mathbb{E}f_i(x_i) -\mathbb{E}F(X))^2P_i$

$= \sum_{i=1}^n \sigma^2(f_i(x_i))P_i + \sum_{i=1}^n (\mathbb{E}f_i(x_i)-\mathbb{E}F(X))^2 P_i$

| cite | improve this answer | |
  • $\begingroup$ The means inside your summation are conditional on the "situation" (event) among the $n$ possible ones, right? $\endgroup$ – DeltaIV Nov 11 '16 at 9:45
  • $\begingroup$ @jbowman Can you explain why the second part is correct? I am having trouble following your reasoning. $\endgroup$ – kilojoules Nov 23 '16 at 22:30
  • $\begingroup$ How can we know we can add the $f(x_i)$ variances with the variance of the expected values? I'm not sure why that would be true. $\endgroup$ – kilojoules Nov 24 '16 at 18:43
  • 1
    $\begingroup$ I've expanded my answer to answer your followup question as well. $\endgroup$ – jbowman Nov 24 '16 at 21:46
  • $\begingroup$ @jbowman I was not able to use your clue to complete this proof. I edited my question to show what I have tried so far and where I'm stuck, $\endgroup$ – kilojoules Nov 28 '16 at 18:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.