Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top


This is one of the theorems in my stats text, and I need some help understanding the proof.

  1. How can the summand($g_{i}$) be out of its summation sign when multiplying?
    I thought you can never take out a summand when $\sum_{i}^{}$ depends on it.
  2. I think there are several steps missing in achieving that final result from the one above.
    Can someone please explain how 2 summations became 1 with a different index?
share|improve this question
up vote 9 down vote accepted

1. The fact that

$$ \sum_{i=1}^m g_i \left\{ \sum_{\substack{\text{all $y_j$ such that} \\ g(y_j)=g_i}} p(y_j) \right\} = \sum_{i=1}^m \sum_{\substack{\text{all $y_j$ such that} \\ g(y_j)=g_i}} g_i p(y_j) $$

is just the distributive property of the sum. Here is a more transparent example: \begin{align*} & \sum_i \left\{ a_i \sum_j b_j \right\} \\ & = \sum_i \left\{ \vphantom{\sum_i} a_i \; (b_1 + b_2 + \dotsb)\right\} \\ & = \sum_i \left\{ \vphantom{\sum_i} (a_i b_1 + a_i b_2 + \dotsb) \right\} \\ & = \sum_i \left\{ \sum_j a_ib_j\right\} \end{align*}


$$ \sum_{i=1}^m \sum_{\substack{\text{all $y_j$ such that} \\ g(y_j)=g_i}} g_i p(y_j) = \sum_{j=1}^n g(y_j) p(y_j) $$

As the 2nd sum is over all $y_j$ such that $g(y_j) = g_i$, you can replace $g_i$ by $g(y_j)$ in its summand:

$$ \sum_{i=1}^m \sum_{\substack{\text{all $y_j$ such that} \\ g(y_j)=g_i}} g_i p(y_j) = \sum_{i=1}^m \sum_{\substack{\text{all $y_j$ such that} \\ g(y_j)=g_i}} g(y_j) p(y_j) $$

Note that $i$ only appears in the index of the 2nd sum. For $i=1$, the 2nd sum is over all $j$'s such that $g(y_j)=g_1$. For $i=2$, the 2nd sum is over all $j$'s (different from before) such that $g(y_j)=g_2$. Etc. Thus, you can replace the double summation by a single summation running over all $j$'s.

share|improve this answer
Awesome, the examples on 1 and 2 really hit the spot! Thank you so much for your explanation!! – Belphegor Feb 22 '14 at 5:22
You're welcome :-) – ocram Feb 22 '14 at 5:26

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.