When mathematical statistics outsmarts probability theory This is not a question, but it is too good to pass. I read it is originally due to Enis, Peter. "On the relation $E (X) = E [E (X∣ Y)]$." Biometrika 60, no. 2 (1973): 432-433.
Assume $Y$ has a Chi-square distribution, with one degree of freedom, so its density is
$$f_Y(y) = \frac{1}{\sqrt{2\pi}}y^{-1/2}\exp\left\{-\frac 12 y\right\},\;\;\; y \in [0,\infty). \tag{1}$$
Suppose now that we define the conditional density of another variable $X$ as
$$f_{X|Y}(x \mid y) = \frac{1}{\sqrt{2\pi}} y^{1/2}\exp\left\{-\frac 12yx^2\right\}\;\;\; x \in (-\infty, \infty). \tag{2}$$
Namely, conditional on $Y=y$, $X$ has a zero-mean Normal distribution with variance equal to $1/y$.
This is perfectly legal and valid. Note that we have
$$E(X\mid Y) = 0.$$
Then "automatically", we would conclude
$$...\implies E\big[E(X\mid Y)\big] = E(X) = 0...$$
BUT THIS IS NOT THE CASE.
By Bayes theorem for densities, the joint density is
$$f_{X,Y}(x,y) = f_{X|Y}(x \mid y)\cdot f_Y(y) = \frac{1}{2\pi}\exp\left\{-\frac 12y(1+x^2)\right\}\;\;\; (x,y) \in (-\infty, \infty) \times (0, \infty). \tag{3}$$
From this we can obtain the marginal density of $X$ by
$$f_X(x) = \int_0^{\infty}f_{X,Y}(x,y) dy = \int_0^{\infty}\frac{1}{2\pi}\exp\left\{-\frac 12y(1+x^2)\right\}\,dy$$
$$=\frac {1}{\pi}\frac{1}{1+x^2}\int_0^{\infty}\frac{1+x^2}{2}\exp\left\{-\frac 12y(1+x^2)\right\}\,dy.$$
The integral is equal to $1$ since it is an Exponential density with rate parameter $(1+x^2)/2$, so we obtain
$$f_X(x) = \frac {1}{\pi}\frac{1}{1+x^2}, \tag{4}$$
which is the standard Cauchy density, for which $E(X)$ is undefined.
So we have obtained that $E(X\mid Y)$ can exist and be finite, even when $E[X]$ does not exist.
Why has mathematical statistics outsmarted probability theory in this instance?
Because in the latter, in order to define the conditional expectation, we start by assuming a random variable $X$ for which $E(X)$ exists, and given this, $E(X\mid Y)$ is then defined -the "defining property" of the conditional expectation in this approach is exactly that its mean equals the unconditional mean. So if we were told, "let $X$ be a standard Cauchy density", we would conclude that "it follows that we cannot define a conditional expectation of it"... But we just saw that this premise of the existence of $E(X)$ is not necessary for $E(X\mid Y)$ to exist...
...the dark side of this achievement, is that it creates the following additional obligation: whenever we encounter first a conditional expectation, we cannot automatically "average over it" to obtain the unconditional expected value -the latter must be proven to exist by other means.
In other words, the existence of the unconditional expected value is only sufficient for the conditional expectation to exist, and the existence of the conditional expectation is not sufficient for the unconditional expected value to exist.
Ah, and here is a question: Any other such lovely examples?
 A: I do not find this any more surprising than saying that if $Y \sim \mathcal N(0,1)$ then $\mathbb E\left[\frac1Y\right]$ is undefined even though $\frac1Y$ has a distribution symmetric about $0$.
So let's use this to construct an example using $Y\sim \mathcal N(0,1)$:

*

*let $X=\pm\frac1Y$ with equal probability (or $0$ in the zero-probability case of $Y=0$)

*clearly $\mathbb E\left[X \mid Y\right]=0$  so $\mathbb E\big[\mathbb E\left[X \mid Y\right]\big]=0$

*but $X$ and $\frac1Y$ have the same heavy-tailed symmetric distribution so $\mathbb E\left[X\right]$ is undefined

A: While it is a pleasant remark, I do not find this occurrence that surprising or paradoxical, and this for several reasons:
(i) $\mathbb E^X[0]=0$ remains true, where $\mathbb E^X[\cdot]$ denotes the expectation under the distribution of $X$;
(ii) conditional distributions and therefore expectations are only defined with respect to or in terms of a joint distribution, meaning that logically we start from this joint and derive the conditional, rather than the opposite, so logically we do not "encounter first a conditional expectation";
(iii) conditional distributions are usually equipped with lighter tails than marginal ones, hence it is not surprising that the conditional expectation may exist for all realisations of $Y$ while the marginal expectation does not exist;
(iv) there is no "defeat of probability theory" there (and even less of a connection with "mathematical statistics"): the law of total expectation states the existence of $\mathbb E[X]$ as its main assumption.
A: 
Note that we have
$$E(X∣Y)=0.$$

$E(X|y) = E[X|1] \cdot y$, which when $y\to \infty$ seems to become like a case of the undefined $0 \times \infty$.
In a way the example poses the naive statement that a distribution must have $E[X] = m$ when it is symmetric about $m$. And it does this via the expression $E (X) = E [E (X∣ Y)]$ which hides the situation that an undefined term is included.
