1
$\begingroup$

The minimzer of the MSE $E[(a-X)^2]$ is $a=E(X)$, and the MSE can be decomposed into $E[(a-X)^2] = E[(a-E(X))^2] + Var(X)$.

I am wondering whether there exists a similar expression th MAE $E[|a-X|]$ in terms of its own minimizer $med(X)$?

Is there a known standard relationship between $E[|a-X|]$, $E[|a-med(X)|]$, and perhaps something like $E[|med(X)-X|]$? Or between $med[|a-X|]$, $med[|a-med(X)|]$, and $med[|med(X)-X|]$? Or anything else that resembles $E[(a-X)^2] = E[(a-E(X))^2] + Var(X)$?

$\endgroup$
7
  • 1
    $\begingroup$ See here: stats.stackexchange.com/questions/7307/… (the arguments there are in terms of sample quantities but essentially the same arguments carry across; there's likely half a dozen other posts on site. Also see here: gregorygundersen.com/blog/2019/10/04/expectation-median-opt $\endgroup$
    – Glen_b
    Commented Dec 28, 2022 at 3:16
  • 1
    $\begingroup$ See stats.stackexchange.com/questions/251600 for a generalization to arbitrary percentiles. There is no such decomposition, though, because it characterizes a Euclidean metric. $\endgroup$
    – whuber
    Commented Dec 28, 2022 at 15:27
  • $\begingroup$ @whuber If you consider the decomposition for the median in my answer is "such a decomposition", then for the general quantiles the similar decomposition should hold as well (see my updated answer). Could you clarify the meaning of "it characterizes a Euclidean metric"? $\endgroup$
    – Zhanxiong
    Commented Dec 28, 2022 at 16:03
  • $\begingroup$ @Zhanxiong It's the Pythagorean Theorem. $\endgroup$
    – whuber
    Commented Dec 28, 2022 at 16:22
  • 1
    $\begingroup$ @whuber Well, I personally think that suffices to answer OP's question (if you check his last paragraph) in that it links $E[|X - a|]$ and $E[|X - m|]$. But I respect your viewpoint as well. $\endgroup$
    – Zhanxiong
    Commented Dec 28, 2022 at 18:32

1 Answer 1

4
$\begingroup$

The short answer to your question is: an analogous decomposition does exist and can be used to show that the minimizer of $\Delta(a) := E[|X - a|]$ is the median (see remarks below for the latter claim).

Denote the median of $X$ by $m$. Using the definition of expectation, we have (note that the essence of the proof, which is shared by the $L^2$ expectation decomposition, is "subtract-then-add"): \begin{align} & E[|X - a|] = \int_{-\infty}^a (a - x)dF(x) + \int_a^{\infty}(x - a)dF(x) \\ =& \int_{-\infty}^m(a - x)dF(x) + \int_m^a(a - x)dF(x) + \int_a^m(x - a)dF(x) + \int_m^\infty(x - a)dF(x) \\ =& \int_{-\infty}^m(m - x)dF(x) + \int_{-\infty}^m(a - m)dF(x) \\ &+ 2\int_a^m(x - a)dF(x) \\ &+ \int_m^\infty(x - m)dF(x) + \int_m^\infty(m - a)dF(x) \\ =& E[|X - m|] + (m - a)(P[X > m] - P[X \leq m]) + 2\int_a^m(x - a)dF(x) \\ =& E[|X - m|] + 2\int_a^m(x - a)dF(x). \end{align} In the penultimate step, we used the condition $P[X \leq m] = P[X > m] = 0.5$. Therefore, the decomposition is \begin{align} E[|X - a|] = E[|X - m|] + 2\int_a^m(x - a)dF(x). \tag{1} \end{align}

Note that the second term in the right hand side of $(1)$, which resembles the term $(E[X] - a)^2$ term in the $L^2$ decomposition, is always non-negative. Under the assumption that $F(x)$ is strictly increasing (so that $m$ is uniquely determined), it is immediate that $\Delta(a)$ is minimized at $a = m$ (for otherwise the integral is strictly positive). When the theoretical median is not unique, the minimizer of $\Delta(a)$ is not unique either.


For a general quantile position $\tau \in (0, 1)$ and the check function $\rho_\tau(u) := u(\tau - I_{(-\infty, 0)}(u))$, the similar decomposition to $(1)$ also holds as follows with the median $m$ replaced with the $\tau$-quantile $q_\tau$ (the proof is almost identical as above):

\begin{align} E[\rho_\tau(X - a)] = E[\rho_\tau(X - q_\tau)] + \int_a^{q_\tau}(x - a)dF(x). \tag{2} \end{align}

Note that $(1)$ and $(2)$ differ by a scaling factor $2$ because $|u| = 2\rho_{0.5}(u)$. Using $(2)$, it is also easy to conclude that $q_\tau = \operatorname{argmin}_{a \in \mathbb{R}^1}E[\rho_\tau(X - a)]$ given the same monotonicity condition of $F$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.