How can I prove mathematically that the mean of a distribution is the measure that minimizes the variance? According to the Page 87 in Kruschke: Doing Bayesian Data Analysis, the author says that the mean of a distribution is a value that minimizes the variance of a probability distribution, for example, a normal distribution. The following is what is mentioned in the page:

It turns out that the value of $ M $ that minimizes $ ∫dx p(x)(x−M)^2$
is $E[X] $. In other words, the mean of the distribution is the value that minimizes the expected squared deviation. In this way, the mean
is a central tendency of the distribution.

I have read the paragraph and kind of understood what the author is trying to say but I wonder how this can be written mathematically using the above equation or any other ways.
 A: Assuming that you have $n$ values $\{x_1, x_2, \ldots, x_n\}$, the mean squared difference from each value $x_i$ to some number $a$ is:
$$
m(a)=\frac{1}{n}\sum_{i=1}^{n}(x_i-a)^2
$$
We could ignore the term $\frac{1}{n}$ but I'm leaving it in. Expand the square and manipulate:
$$
m(a)=\frac{1}{n}\left(\sum_{i=1}^{n}x_{i}^{2}-2ax_i+a^2\right)=\frac{1}{n}\left(\sum_{i=1}^{n}x_{i}^{2}-\sum_{i=1}^{n}2ax_i+\sum_{i=1}^{n}a^2\right)=\frac{1}{n}\left(\sum_{i=1}^{n}x_{i}^{2}-2a\sum_{i=1}^{n}x_i+na^2\right)
$$
To find the value of $a$ that minimizes the expression, differentiate with respect to $a$ to get:
$$
m'(a)=\frac{1}{n}\left(2na - 2\sum_{i=1}^{n}x_i\right)
$$
Now set this to zero an solve for $a$ to get:
$$
a=\frac{\sum_{i=1}^{n}x_i}{n}
$$
which is the mean of the values $x_i$. To show that this is indeed the minimum, calculate the second dervative wrt to $a$, which is $m''(a)=2>0$ so it's a minimum.
A: Budding statisticians in Statistics 101 with no mathematical skills beyond high-school algebra should consider
\begin{align}
E\left[(X-a)^2\right] &= E\bigr[\big(X-\mu + \mu -a\big)^2\bigr] & {\scriptstyle{\text{Here,}~\mu ~ \text{denotes the mean of} ~ X}}\\
&= E\bigr[\big((X-\mu) + (\mu -a)\big)^2\bigr]\\
&= E\bigr[(X-\mu)^2 + (\mu -a)^2 &{\scriptstyle{(\alpha+\beta)^2 = \alpha^2+\beta^2 + 2\beta\alpha}}\\
& ~~~~~~~~~~~~~~+ 2(\mu-a)(X-\mu)\bigr]\\
&= E\big[(X-\mu)^2\big] + E\big[(\mu -a)^2\big]
&{\scriptstyle{\text{Expectation of sum is the sum of}}}\\
&  ~~~~~~~~~+ 2(\mu-a)E\big[X-\mu\big] &{\scriptstyle{\text{the expectations, and constants}}}\\
& &{\scriptstyle{\text{can be pulled out ofexpectations}}}\\
&= \operatorname{var}(X) +  (\mu -a)^2 + 2(\mu-a)\times 0
&{\scriptstyle{\text{definition of variance; expectation}}}\\
& &{\scriptstyle{\text{of a constant equals the constant;}}}\\
& &{\scriptstyle{E[X-\mu] = E[X] -E[\mu] = \mu -\mu = 0}}\\
&= \operatorname{var}(X) +  (\mu -a)^2\\
&\geq \operatorname{var}(X) &{\scriptstyle{\text{with equality holding when}~ a=\mu.}}
\end{align}
Thus, we have shown that $E\left[(X-a)^2\right] \geq \operatorname{var}(X)$ with equality holding when $a = \mu$.
Look, Ma! No calculus! No derivatives or second derivative tests! Not even geometry and invocations of Pythogoras; just high-school algebra (even pre-algebra might have sufficed), together with a just a smidgen of Stats 101.
A: A quick one:
This follows from a property of moments (a rule to transform the center)
$$E\left[(x-\hat{x})^n\right] = \sum_{i=0}^n {n \choose i} E\left[(x-a)^i\right]  (a-\hat{x})^{n-i}$$
which becomes for $n=2$ and $a=\mu=E[X]$
$$E \left[(x-\hat{x})^2\right] =  \underbrace{E \left[(x-\mu)^2\right] }_{=\text{Var}(x)} + 2   \underbrace{E \left[(x-\mu)\right] }_{=0} (\mu-\hat{x}) + (\mu-\hat{x})^2 = \text{Var}(x) + (\mu-\hat{x})^2 $$
and this is minimised when $\hat{x}=\mu$.

A longer one:
It goes similar using the expression $∫dx p(x)(x−M)^2$
$$\begin{array}{}
∫dx p(x)(x−M)^2  &=& ∫dx p(x)\underbrace{((x-\mu)+(\mu-M))^2}_{=(x-\mu)^2 + 2(x-\mu)(\mu-M) + (\mu-M)^2}  \\
& =& ∫dx p(x)(x-\mu)^2 + ∫dx p(x)2(x-\mu)(\mu-M) +∫dx p(x) (\mu-M)^2 \\
& =& ∫dx p(x)(x-\mu)^2 + ∫dx p(x)2 x(\mu-M) - ∫dx p(x)2\mu(\mu-M)+∫dx p(x) (\mu-M)^2 \\
& =& \underbrace{∫dx p(x)(x-\mu)^2}_{=\text{var}(x)} + 2 (\mu-M)\underbrace{∫dx p(x)x}_{=\mu} - 2 (\mu-M)\mu \underbrace{∫dx p(x)}_{=1}+(\mu-M)^2 \underbrace{∫dx p(x)}_{=1}  \\
&=& \text{var}(x) +(\mu-M)^2
\end{array}$$

Intuitively:
In other words, the variance is the 2nd moment about the mean and a 2nd moment about some other point will be larger.
In order to find the optimum you could differentiate the integral and set equal to zero then you get
$$\frac{\partial}{\partial M} ∫dx p(x)(x−M)^2  =2 ∫dx p(x)(x−M) = 2(\mu-M) =0 $$
Which results in $M=\mu$.
In words: if you shift the point $M$ then the contributions of the $(x-M)^2$ term change and become less or more, and this is more when the distance $x-M$ is larger. When $M$ is equal to the mean $\mu$ the increase and decrease (the sum/mean of $x-M$) balance each other and you reach the minimum.
Something similar can be done to minimize the integral $∫dx p(x)|x−M|$ and you will find that this is minimised when $M$ is equal to the median.
A: Just work it out.
$$\frac{\partial}{\partial c}E[(X-c)^2]=E\left[ \frac{\partial}{\partial c}(X-c)^2 \right]=2E\left[ X-c \right]$$
Setting this to zero, $E[X]=c$ is the only stationary point and it's obviously not a maximum.
.
(there's an invocation of the dominated convergence theorem or near offer in there to justify putting the derivative through the integral, which might fail if the variance doesn't exist or something pathological like that)
A: The expression is $\mathbb E[(X-a)^2]$. We'll differentiate and equate the expression to $0$:
$$\begin{align}\frac{d}{da}\mathbb E[(X-a)^2]&=\mathbb E\left[\frac{d}{da}(X-a)^2\right]\\&=\mathbb E[-2(X-a)]\\&=0\end{align}$$
Then, $\mathbb E[-2X +2a]=0\rightarrow \mathbb E[2X]=\mathbb E[2a]=2a\rightarrow a=\mathbb E[X]$. The second derivative is positive, so it's the minimizer.
A: I have another geometric perspective which my college suggested me. In essence "the value $M$" that you are looking for is a factor such that $M\cdot\mathbb{1}$ is a projection of $X$ onto subspace generated by $\mathbb{1}$ in $L^2(\Omega, A, P)$ space where $\mathbb{1}$ is a constant random variable equal 1 almost everywhere.
Recall that if we have an $\mathbb{R}^n$ space with an inner product $\langle\cdot,\cdot\rangle$ and we want to project vector $p$ on subspace generated by vector $x$ then the projected vector is given by formula
$$v=\frac{\langle p,x\rangle}{||x||^2}x$$
and $v$ it is a unique vector of form $Mx$ which minimizes $||p-Mx||^2.$
Instead of regular $\mathbb{R}^n$ consider space $L^2(\Omega, A, P)$ (space of random variables $X:\Omega\to\mathbb{R}$ such that $\operatorname{E}[X^2]<\infty$) with an inner product given by formula
$$\langle X_1,X_2\rangle = \operatorname{E}[X_1\cdot X_2]=\int_\Omega X_1(x)X_2(x)dP(x).$$
If we fix an arbitrary random variable $X$ from $L^2(\Omega, A, P)$ and we want to project it onto subspace generated by random variable $\mathbb{1}$, then it is the same of finding $M$ such that it minimizes
$$ \operatorname{E}[(X-M\cdot\mathbb{1})^2].$$
Using the same argument as in the $\mathbb{R}^n$, we get that $M=\frac{\langle X,\mathbb{1}\rangle}{||\mathbb{1}||^2}=\operatorname{E}[X]$.
