4
$\begingroup$

Let $X\sim F(x)$ be a (univariate) random variable defined by distribution function $F$. If the expected value exists, it is equal to $ \mathbb E[X] = \underset{\mu}{\arg\min}\left\{\mathbb E\left\vert X-\mu\right\vert^2\right\} $.

However, the expected value does not have to exist.

What conditions are there on the exponent $p$ such that $ \underset{\mu}{\arg\min}\left\{\mathbb E\left\vert X-\mu\right\vert^p\right\} $ must exist? How low do we have to go before it must exist? We trivially get the $\arg\min$ equal to $\mathbb R$ if $p=0$ and get the median as the $\arg\min$ for $p=1$. What about $p=1.5$ or $1.9$ or $1.99$?

I wonder if $2$ is the supremum, but not in the set, of $p$ that guarantee an $\arg\min$, and that any $p<2$ must have a defined $\arg\min$. If there is interesting commentary about uniqueness, I am on board with going down that rabbit hole.

My inspiration came from here, and I wonder if a reasonable answer to the linked question is to lower $p$ until the $\arg\min$ must exist.

$\endgroup$
11
  • $\begingroup$ Can you say a little more about what you mean when you say that $\textrm{argmin}...$ exists? I imagine you mean that the set of minimizers has one and only one member (for example: it is given by $\mathbb{R}$ if $p=0$)? And also: are you sure you don't want to take an absolute value before exponentiation, i.e. $|X-\mu|^p$? relevant for e.g. $p=3$. Anyways, awesome question and I can't wait to see the answers :) $\endgroup$ Commented Mar 31 at 23:04
  • $\begingroup$ @JohnMadden I have made some edits that I hope clarify my question. I’m really interested in lowering $p$ below $2$ until the $\arg\min$ must exist. I have my doubts that any $p>2$ requires the $\arg\min$ to exist (but am open to learning why that is wrong if it is). $\endgroup$
    – Dave
    Commented Mar 31 at 23:23
  • 1
    $\begingroup$ "$\mathbb E[X] = \underset{\mu}{\arg\min}\left\{\left\vert X-\mu\right\vert^2\right\}$" -- Wrong. The correct statement is $\mathbb E[X] = \underset{\mu}{\arg\min}\color{red}{E}\left\{\left\vert X-\mu\right\vert^2\right\}$. $\endgroup$
    – Zhanxiong
    Commented Apr 1 at 0:44
  • 2
    $\begingroup$ For the symmetric stable distributions, $p=1$ corresponds to the Cauchy, which leads to the median, as you've noted, but for $p<1$, say, $1/2$, does the expectation $|X-\mu|^{1/2}$ always exist? I doubt it, but am not in a position to work out an example ATM. $\endgroup$
    – jbowman
    Commented Apr 1 at 2:02
  • 1
    $\begingroup$ @picky_porpoise I think a proof of that is exactly what I want! $\endgroup$
    – Dave
    Commented Apr 1 at 13:59

2 Answers 2

5
$\begingroup$

The choice $p=0$ is the only one such that the arg min in question always exists. For any $p>0$ we can find a distribution $F$ such that $\mathbb{E} \vert X - \mu \vert^p$ does not exist for any $\mu \in \mathbb{R}$ and thus the arg min is empty.

The simplest way to see this, is to take the Pareto distribution with parameter $\alpha > 0$ on $[1, \infty)$ for $F$. Its density is given by $$ f(x) = \frac{\alpha}{x^{\alpha + 1}} $$ for $x > 1$. From basic analysis we can see that the $p$-th moment $\mathbb{E}\vert X \vert^p$ of an $\alpha$-Pareto distribution exists only if $\alpha > p$ and as pointed out in the answer by jacques, the existence of $\mathbb{E} \vert X - \mu \vert^p$ is equivalent to the existence of $\mathbb{E} \vert X \vert^p$. Hence, $\underset{\mu}{\arg\min} \, \mathbb{E}\left\vert X-\mu\right\vert^p $ is empty for a sufficiently heavy-tailed Pareto distribution.

$\endgroup$
2
$\begingroup$

Generally, for $E[\vert X-\mu \vert^p]$ to be finite, it is necessary and sufficient that $E[\vert X \vert^p] < \infty$, i.e. that $X \in L^p$ (I think you forgot the expectation in your question), because you can estimate $\vert X - \mu\vert^p \leq 2^p (\vert X \vert^p + \vert\mu\vert^p)$ (and for the other direction use $X = (X - \mu ) + \mu$). In that case, also the arg-min will be well-defined.

Generally, for finite measure spaces, and in particular for probability spaces (where the total mass is 1), $L^p \subset L^q$ for $p > q$ (this is a consequence of Jensen's inequality applied to the function $\varphi(x)=\vert x \vert^{p/q}$... although it can also be proven using Hölder's inequality applied to $f = \vert X \vert^q$ and $g = 1$). In general, the characterization of the median/mean of a random variable in terms of its minimizing $E \vert X - \mu \vert $ and $E[(X - \mu)^2]$ is not valid in general (e.g. mean exists if $X \in L^1$, but the criterion is well-defined if $X \in L^2$ [example: Pareto distribution for $\alpha \in (1, 2)$ has finite mean but infinite variance, hence the criterion is not well-defined]... moreover, the median always exists, but the criterion is only well-defined if $X \in L^1$ [example: Cauchy distribution]).

$\endgroup$
2
  • $\begingroup$ What do you mean in that last comment about the median? $\endgroup$
    – Dave
    Commented Apr 1 at 2:41
  • 3
    $\begingroup$ @Dave The median exists for all distributions, but only for the ones with finite mean can it be represented as the arg min of the expected absolute distance. $\endgroup$ Commented Apr 1 at 12:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.