Conditional Expectation of Multivariate Distributions My book states that $\mathrm{E}[X_2] = \mathrm{E}[\mathrm{E}[X_2\mid X_1]]$ and gives a weird proof and a short paragraph about how awesome this result is and how intuitive this is.
I neither understand the proof nor do I understand the "intuition". Can someone give me both? 
The textbook proof uses some non-standard notation and weird transformations but I am hoping for a clear lucid proof.
 A: Try to build your intuition with particular cases first. Suppose that $X\in\{x_1,\dots,x_m\}$ with pmf $p_X$ and $Y\in\{y_1,\dots,y_n\}$ with pmf $p_Y$, such that $p_Y(y_i)>0$, for $i=1,\dots,n$. Let $p_{X,Y}$ be the joint pmf of $X$ and $Y$. The conditional pmf of $X$ given $Y$ is defined as
$$
  p_{X\mid Y}(x_i\mid y_j) = \frac{p_{X,Y}(x_i,y_j)}{p_Y(y_j)} \qquad . \qquad\qquad (*)
$$
Note that $(*)$ is a pmf for each "fixed" $y_j$. Now, interpret the expectation
$$
  \mathrm{E}[X]=\sum_{i=1}^m x_i\,p_X(x_i)
$$
intuitively as your best guess about the value of $X$, and extend this interpretation to a new object $\mathrm{E}[X\mid Y=y]$, called the conditional expectation of $X$ given that $Y=y$, which represents your best guess about the value of $X$ when you are given the information that $Y=y$. 
The key point is that this conditional expectation is computed from $(*)$ as
$$
  \mathrm{E}[X\mid Y=y] = \sum_{i=1}^m x_i\,p_{X\mid Y}(x_i\mid y) = g(y) \, ,
$$
for some nice function $g$. The notation $\mathrm{E}[X\mid Y]$ is just an abreviation for the random variable $g(Y)$ (that is what I've asked you in my comment). But you already now how to compute the expectation of $g(Y)$ using the "law of the unconscious statistician" as
$$
  \mathrm{E}[g(Y)] = \sum_{i=1}^n g(y_i)\,p_Y(y_i) = \sum_{i=1}^n \mathrm{E}[X\mid Y=y_i]\,p_Y(y_i)
$$
$$ 
  = \sum_{i=1}^n \sum_{j=1}^m x_j\,p_{X\mid Y}(x_j\mid y_i)\,p_Y(y_i)
$$
$$
  = \sum_{j=1}^m x_j \sum_{i=1}^n p_{X,Y}(x_j,y_i) = \sum_{j=1}^m x_j\, p_X(x_j)
$$
which can be written as
$$
  \mathrm{E}[\mathrm{E}[X\mid Y]]=\mathrm{E}[X] \, .
$$
The importance of this concept is impossible to overstate. Its generalization to the conditional expectation given a sigma-field dominates the development of modern probability theory, martingale theory, etc. Defining conditional variance $\mathrm{Var}[X\mid Y]$ in the same spirit, the corresponding property
$$
  \mathrm{Var}[X] = \mathrm{E}[\mathrm{Var}[X\mid Y]] + \mathrm{Var}[\mathrm{E}[X\mid Y]] 
$$
is the basis for a variance reduction technique extensively used in simulation known as rao-blackwelization.
To give you a taste of the conditional expectation as a tool, consider this problem. Let $N$ be the number of questions posted monthly at Stack Exchange, and let $X_i$ be the number of answers to question $i$. Suppose that the $X_i$'s are IID, and that $N$ and the $X_i$'s are independent. The number of total answers in a month is given by the random sum 
$\sum_{i=1}^N X_i$ (note that the number of terms in this sum is random). What is $\mathrm{E}\left[\sum_{i=1}^N X_i\right]$?
To solve this we will justify some properties of the conditional expectation intuitively. First, we "use what we know", plus the independence of the $X_i$'s and $N$ to get
$$
  \mathrm{E}\left[\sum_{i=1}^N X_i \,\Bigg|\, N = n\right] = \mathrm{E}\left[\sum_{i=1}^n X_i \,\Bigg|\, N = n\right] = \mathrm{E}\left[\sum_{i=1}^n X_i\right] = n \,\mathrm{E}\left[X_1\right] \, .
$$
Hence, the random variable $\mathrm{E}\left[\sum_{i=1}^N X_i\mid N\right]=N \,\mathrm{E}\left[X_1\right]$, and using the fundamental property we have
$$
  \mathrm{E}\left[\sum_{i=1}^N X_i\right]=\mathrm{E}\left[\mathrm{E}\left[\sum_{i=1}^N X_i \,\Bigg|\, N\right]\right] = \mathrm{E}[N]\, \mathrm{E}\left[X_1\right] \, .
$$
