14
$\begingroup$

I have a 1-D random variable which is extremely skewed. In order to normalize this distribution, I want to use the median rather than the mean. my question is this: can I calculate the variance of the distribution using the median in the formula instead of the mean?

i.e. can I replace

$$ \mathrm{Var}(X) = \sum[(X_i - \mathrm{mean}(X))^2]/n $$

with

$$ \mathrm{Var}(X) = \sum[(X_i - \mathrm{median}(X))^2]/n $$

My reasoning behind this is that since variance is a measure of spread w.r.t. the central tendency of a distribution, it shouldn't be a problem but I'm looking to validate this logic.

Improve this question
$\endgroup$
10
  • 2
    $\begingroup$ See en.wikipedia.org/wiki/Median_absolute_deviation $\endgroup$
    – Tim ♦
    Oct 15, 2015 at 12:43
  • 1
    $\begingroup$ By median centering your variables and then dividing that by the MAD (median absolute deviation), you can create a median standardized distribution. $\endgroup$
    – Mike Hunter
    Oct 15, 2015 at 12:47
  • 5
    $\begingroup$ You can do this! But I think it's fair to call it highly non-standard and to suggest that you need theory and/or simulations to back it up and not just your intuition. I suspect that it will be less resistant than the standard estimator. For example, in a common right-skewed case, the median will be less than the mean, so the largest squared deviations (from the median) will therefore be even larger! The major point is that if the variance is very untrustworthy, you may need to think about measuring spread quite differently, rather than different versions of the variance. $\endgroup$
    – Nick Cox
    Oct 15, 2015 at 12:53
  • 1
    $\begingroup$ Orthogonal point: Does "normalise" mean scale in some way, e.g. (value $-$ location) / scale, or does it mean make closer to normal (Gaussian)? $\endgroup$
    – Nick Cox
    Oct 15, 2015 at 12:56
  • 1
    $\begingroup$ This approach is inherently inconsistent, because the problems that are addressed by replacing the mean by the median are magnified by using the variance instead of a robust estimator of the spread. $\endgroup$
    – whuber ♦
    Oct 15, 2015 at 14:21

1 Answer 1

Reset to default
13
$\begingroup$

Mean minimizes the squared error (or L2 norm, see here or here), so natural choice for variance to measure distance from the mean is to use squared error (see here on why we square it). On the other hand, median minimizes the absolute error (L1 norm), i.e. it is a value that is in the "middle" of your data, so absolute distance from the median (so called Median Absolute Deviation or MAD) seems to be a better measure of the degree of variability around the median. You can read more about this relations in this thread.

Saying it short, variance differs from MAD on how do they define the central point of your data and this influences the way how we measure variation of datapoints around it. Squaring the values make outliers have greater influence on the central point (mean), while in case of median, all the points have the same impact on it, so the absolute distance seems more appropriate.

This can be shown also by simple simulation. If you compare values squared distances from the mean and median, then the total squared distance is almost always smaller from mean than from median. On the other hand, total absolute distance is smaller from median, then from mean. The R code for conducting the simulation is posted below.

sqtest  <- function(x) sum((x-mean(x))^2)  < sum((x-median(x))^2)
abstest <- function(x) sum(abs(x-mean(x))) > sum(abs(x-median(x)))

mean(replicate(1000, sqtest(rnorm(1000))))
mean(replicate(1000, abstest(rnorm(1000))))

mean(replicate(1000, sqtest(rexp(1000))))
mean(replicate(1000, abstest(rexp(1000))))

mean(replicate(1000, sqtest(runif(1000))))
mean(replicate(1000, abstest(runif(1000))))

In the case of using median instead of mean in estimating such "variance" this would lead to higher estimates, than with using mean as it is done traditionally.

By the way, the relations of L1 and L2 norms can be considered also in the Bayesian context, as in this thread.

Improve this answer
$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.