38
$\begingroup$

If so, what? If not, why not?

For a sample on the line, the median minimizes the total absolute deviation. It would seem natural to extend the definition to R2, etc., but I've never seen it. But then, I've been out in left field for a long time.

$\endgroup$
1

7 Answers 7

21
$\begingroup$

I'm not sure there is one accepted definition for a multivariate median. The one I'm familiar with is Oja's median point, which minimizes the sum of volumes of simplices formed over subsets of points. (See the link for a technical definition.)

Update: The site referenced for the Oja definition above also has a nice paper covering a number of definitions of a multivariate median:

$\endgroup$
2
17
$\begingroup$

As @Ars said there are no accepted definition (and this is a good point). There are general alternatives families of ways to generalize quantiles on $\mathbb{R}^d$, I think the most significant are:

  • Generalize quantile process Let $P_n(A)$ be the empirical measure (=the proportion of observations in $A$). Then, with $\mathbb{A}$ a well chosen subset of the Borel sets in $\mathbb{R}^d$ and $\lambda$ a real valued measure, you can define the empirical quantile function:

$U_n(t)=\inf (\lambda(A) : P_n(A)\geq t A\in\mathbb{A})$

Suppose you can find one $A_{t}$ that gives you the minimum. Then the set (or an element of the set) $A_{1/2-\epsilon}\cap A_{1/2+\epsilon}$ gives you the median when $\epsilon$ is made small enough. The definition of the median is recovered when using $\mathbb{A}=(]-\infty,x] x\in\mathbb{R})$ and $\lambda(]-\infty,x])=x$. Ars answer falls into that framework I guess... tukey's half space location may be obtained using $\mathbb{A}(a)=( H_{x}=(t\in \mathbb{R}^d :\; \langle a, t \rangle \leq x ) $ and $\lambda(H_{x})=x$ (with $x\in \mathbb{R}$, $a\in\mathbb{R}^d$).

  • variational definition and M-estimation The idea here is that the $\alpha$-quantile $Q_{\alpha}$ of a random variable $Y$ in $\mathbb{R}$ can be defined through a variational equality.

  • The most common definition is using the quantile regression function $\rho_{\alpha}$ (also known as pinball loss, guess why ? ) $Q_{\alpha}=arg\inf_{x\in \mathbb{R}}\mathbb{E}[\rho_{\alpha}(Y-x)]$. The case $\alpha=1/2$ gives $\rho_{1/2}(y)=|y|$ and you can generalize that to higher dimension using $l^1$ distances as done in @Srikant Answer. This is theoretical median but gives you empirical median if you replace expectation by empirical expectation (mean).

  • But Kolshinskii proposes to use Legendre-Fenchel transform: since $Q_{\alpha}=Arg\sup_s (s\alpha-f(s))$ where $f(s)=\frac{1}{2}\mathbb{E} [|s-Y|-|Y|+s]$ for $s\in \mathbb{R}$. He gives a lot of deep reasons for that (see the paper ;)). Generalizing this to higher dimensions require working with a vectorial $\alpha$ and replacing $s\alpha$ by $\langle s,\alpha\rangle$ but you can take $\alpha=(1/2,\dots,1/2)$.

  • Partial ordering You can generalize the definition of quantiles in $\mathbb{R}^d$ as soon as you can create a partial order (with equivalence classes).

Obviously there are bridges between the different formulations. They are not all obvious...

$\endgroup$
1
  • $\begingroup$ Nice answer, Robin! $\endgroup$
    – ars
    Aug 20, 2010 at 18:07
12
$\begingroup$

There are distinct ways to generalize the concept of median to higher dimensions. One not yet mentioned, but which was proposed long ago, is to construct a convex hull, peel it away, and iterate for as long as you can: what's left in the last hull is a set of points that are all candidates to be "medians."

"Head-banging" is another more recent attempt (c. 1980) to construct a robust center to a 2D point cloud. (The link is to documentation and software available at the US National Cancer Institute.)

The principal reason why there are multiple distinct generalizations and no one obvious solution is that R1 can be ordered but R2, R3, ... cannot be.

$\endgroup$
8
  • $\begingroup$ Any measure that coincides with the usual median when restricted to R1 is a candidate generalization. There must be a lot of them. $\endgroup$
    – phv3773
    Aug 24, 2010 at 19:37
  • $\begingroup$ phv:> one can ask for 'the' generalization to preserve (in higher dimensions) some of the interesting properties of the median. This severly limits the number of candidates (see the commenting after Srikant's answer below) $\endgroup$
    – user603
    Sep 20, 2010 at 20:10
  • 1
    $\begingroup$ @Whuber:> You state: "R1 can be ordered but R2, R3, ... cannot be". R2,..,R3 can be ordered in many ways by mapping from Rn to R . One such way is the tukey depth. It has many important properties (robustness to some extend, non parametric, invariance,...) but these only hold for the case of unimodal distributions. Let me know if you want more details. $\endgroup$
    – user603
    Sep 21, 2010 at 1:47
  • 2
    $\begingroup$ @Kwak: A continuous mapping from R^n to R cannot induce a total order. Technically, I was wrong, because any bijection from R^n to R will "order" R^n via the order on R, but this order will not be compatible with any of the metric structure on R^n (which is an essential part of the concept of a "median"), and that was the spirit of my comment. $\endgroup$
    – whuber
    Sep 28, 2010 at 22:31
  • 1
    $\begingroup$ @Silverfish Thank you for noticing this. Fortunately the material is still available, but was relocated. I updated the link. In the future we should all make an effort to accompany external links with enough information to recover them if they get broken. $\endgroup$
    – whuber
    Feb 11, 2015 at 14:38
10
$\begingroup$

Geometric median is the point with the smallest average euclidian distance from the samples

$\endgroup$
1
6
$\begingroup$

The Tukey halfspace median can be extended to >2 dimensions using DEEPLOC, an algorithm due to Struyf and Rousseeuw; see here for details.

The algorithm is used to approximate the point of greatest depth efficiently; naive methods which attempt to determine this exactly usually run afoul of (the computational version of) "the curse of dimensionality", where the runtime required to calculate a statistic grows exponentially with the number of dimensions of the space.

$\endgroup$
4
$\begingroup$

A definition that comes close to it, for unimodal distributions, is the tukey halfspace median

$\endgroup$
1
$\begingroup$

I do not know if any such definition exists but I will try and extend the standard definition of the median to $R^2$. I will use the following notation:

$X$, $Y$: the random variables associated with the two dimensions.

$m_x$, $m_y$: the corresponding medians.

$f(x,y)$: the joint pdf for our random variables

To extend the definition of the median to $R^2$, we choose $m_x$ and $m_y$ to minimize the following:

$E(|(x,y) - (m_x,m_y)|$

The problem now is that we need a definition for what we mean by:

$|(x,y) - (m_x,m_y)|$

The above is in a sense a distance metric and several possible candidate definitions are possible.

Eucliedan Metric

$|(x,y) - (m_x,m_y)| = \sqrt{(x-m_x)^2 + (y-m_y)^2}$

Computing the median under the euclidean metric will require computing the expectation of the above with respect to the joint density $f(x,y)$.

Taxicab Metric

$|(x,y) - (m_x,m_y)| = |x-m_x| + |y-m_y|$

Computing the median in the case of the taxicab metric involves computing the median of $X$ and $Y$ separately as the metric is separable in $x$ and $y$.

$\endgroup$
8
  • $\begingroup$ Srikant:> No. The definition has to have two important feature of the univariate median. a) Invariant to monotone transformation of the data, b) robust to contamination by outliers. None of the extentions you propose have these. The Tukey depth has these qualities. $\endgroup$
    – user603
    Sep 20, 2010 at 16:57
  • $\begingroup$ @kwak What you say makes sense. $\endgroup$
    – user28
    Sep 20, 2010 at 18:25
  • $\begingroup$ @Srikant:> Check the R&S paper cited by Gary Campbell above ;). Best, $\endgroup$
    – user603
    Sep 20, 2010 at 18:38
  • $\begingroup$ @kwak On thinking some more, the taxicab metric does have the features you mentioned as it basically reduces to univariate medians. no? $\endgroup$
    – user28
    Sep 20, 2010 at 18:40
  • 2
    $\begingroup$ @Srikant:> there are no incorrect answer to phv's questions because there are no 'good answers' either; this area of research is still under development. I simply wanted to point out why it is still an open problem. $\endgroup$
    – user603
    Sep 20, 2010 at 20:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.