Error bars using median absolute deviation

Question

I have tried to find a solution to my question on Google, but I can’t seem to find much information about error bars and median absolute deviation and I do not know much about statistical error analysis so any help would be greatly appreciated.

I am creating a semi-log plot for my astronomy research that splits the data into 5 equally spaced bins in log base 10 (x-axis) and plots the average value for each bin (y-axis). Thus each bin has a larger sample size as the x-axis increases. Since it’s a statistical study there is a lot of potential for uncertainty and there is a high presence of outliers (non-Gaussian error distributions). As a result, I would like to use median absolute deviation (MAD) error bars since MAD is less susceptible to outliers.

So my question is: Much like standard deviation and standard error, where standard error is just stdev/sqrt(N) and N is the size of the sample – is there an analog to this for median absolute deviation? The error bars I get when I use MAD/sqrt(N) look correct, but I am not confident in explaining my reasoning for using this approach. Is dividing MAD by the square root of the sample size an acceptable means to produce error bars? And if not, do you have any other suggestions? Also, from research I’ve done it seems that median absolute deviation is a better estimator than mean absolute deviation…would you agree with this?

UPDATE:

Here is my original plot with error bars from median absolute deviation/sqrt(N) (NOTE: title should read median, not mean!) The first bin contains 39 samples, second contains 146, third 454, fourth 1287, and fifth 2371 samples. It looks nice, but the method for producing error bars does not seem very accurate.

I've created error bars using the bootstrap method as suggested by @Glen_b (attached below). It looks good to me, but my advisor feels that the error bars have been overestimated in this method.

Answer 1

It sounds like you're talking about what's sometimes called a regressogram, with a log-scaled x-variable.

There are a number of issues here, not necessarily in logical order:

• the quantity you're plotting is a mean, so if you want to plot median absolute deviation, it's the MAD of the means you want.

• your suggestion $\text{MAD}/\sqrt n$ leads to the question "when is the MAD of the mean equal to the MAD of the data divided by $\sqrt n$?"

• when you say "it seems that median absolute deviation is a better estimator than mean absolute deviation" ... that depends what we're talking about - a better estimator of what?, and under what circumstances?

---

So, "when is the MAD of the mean equal to the MAD of the data divided by $\sqrt n$?"

The answer is, unlike the situation with standard deviation, this is not generally the case. The reason why standard deviations of averages scale as they do is that variances of independent random variables add (more precisely, the variance of the sum is the sum of the variances when the variables are independent), irrespective of the distributions of the components (as long as the variances all exist). It is this particular property that largely accounts for the popularity of variances and standard deviations.

Neither the median deviation, nor the mean deviation have that property in general.

However, when the data are normal, they will in effect inherit that property, since the ratio of the population mean deviation or median deviation to the standard deviation at a normal will be a constant, normals are closed under convolution, and standard deviations scale that way.

If the data were reasonably close to normal, it could perhaps be adequate.

---

What else might be done? One way to estimate the standard error of a statistic is via the bootstrap; for the mean deviation - being a mean - this should do well in large samples. Unfortunately, medians don't do so well under the bootstrap, and this issue will carry over to median absolute deviations.

If you have some probability model for your data, there's also simulation as a way of approaching the problem.

Answer 2

A standard error means something. You don't just take any old statistic and divide by sqrt(n). Why not just plot your MAD and have your error bar a representation of variability in the data? If you want something to represent the quality of your median estimate then just calculate a confidence interval of the median.

Answer 3

Whatever you do, plot your raw data or minimally make them available some how.

If you choose median absolute deviation (MAD), do make it absolutely clear whether it is of deviations from the mean or the median, as I've seen MAD used as an abbreviation for both and in any case any ambiguity benefits no-one.

Plotting +/- MAD as error bars has a loose connection to the widely used box plots in which median and quartiles are shown in a box and there are various different recipes for what is shown outside the box.

MAD is approximately |quartile $-$ median| in a symmetric distribution. For a symmetric distribution it's immaterial whether MAD is MAD from median and or from mean or "quartile" is the upper or lower quartile. MAD will be similar to (upper q. $-$ median) and (median $-$ lower q.) even in many asymmetric distributions. There are various slightly different rules for quartiles, which may cause minor puzzles, but is not central here.

A bigger question is this: if outliers make your standard errors dubious, how come you want to show means, because they will be affected too? As @John implies, a median is clearly a possibility. Also, would you be better off on a logarithmic or other transformed scale for your y variable too?