2
$\begingroup$

I have a pair of data sets, and I want to know whether the medians of these two sets differ significantly. I've calculated the median of each set and subtracted them from each other. How can I determine appropriate error bars for that point?

I was thinking I would take the median absolute deviation (MAD) of both and add them together. But should that be the entire length of the error bar? Or half of it? Or should I divide it by $\sqrt n$? Any advice is appreciated, thanks.

$\endgroup$
6
  • 2
    $\begingroup$ Neither. You would need an estimate of the standard error of the difference of the medians, and that is complicated. (It depends on the type of distribution, among other things.) Meanwhile, several nonparametric tests, among them the Mann-Whitney and the sign test, are very well suited for comparing locations of two distributions with the same shape. $\endgroup$
    – Russ Lenth
    Jul 28, 2014 at 16:46
  • 1
    $\begingroup$ PS to get the error bar from one of these tests, find the lowest and smallest values of $a$ (could be negative or positive) that you could add to one sample's values and obtain a barely nonsignificant test result. Some M-W software will also output a confidence interval based on that idea. $\endgroup$
    – Russ Lenth
    Jul 28, 2014 at 16:50
  • $\begingroup$ Mann-Whitney helped me a lot, thanks. You can submit that as an answer and I'll accept it. $\endgroup$ Jul 28, 2014 at 18:55
  • $\begingroup$ Glad it was helpful. Before I do anything like that, I want to check. I'm fairly new to Stack Exchange, and wonder if it is a welcome practice to resubmit a comment as an answer. Can somebody advise? $\endgroup$
    – Russ Lenth
    Jul 28, 2014 at 19:16
  • $\begingroup$ For what it's worth, I've been here many years and I see that happen all the time. $\endgroup$ Jul 28, 2014 at 20:01

2 Answers 2

2
$\begingroup$

Previous comment regenerated as an answer...

I suggest using the Mann-Whitney test, which is very well suited for comparing locations of two distributions with the same shape. To get a confidence interval for the difference of medians, find the lowest and smallest values of a constant $a$ (could be negative or positive) that, if added to Sample #1's values, you'll obtain a barely nonsignificant test result. Some M-W software will already output a confidence interval based on that idea.

$\endgroup$
2
  • 1
    $\begingroup$ +1 However it should be noted that the Mann-Whitney will give a CI for the population Hodges-Lehmann location-difference (median of the distribution of cross-group pairwise differences, a.k.a. the pseudomedian difference). It would be a proper CI for the difference in population medians if the two population distributions have the same shape and spread -- but then it's equally a CI for the difference in means, modes, tenth percentiles, midmeans or whatever you like (as long as the population quantity exists and is finite). $\endgroup$
    – Glen_b
    Jul 28, 2014 at 23:35
  • $\begingroup$ Actually I'm deleting the comment that had a link on the population location parameter the Mann-Whitney gives a CI for. It discussed it but the discussion is ... well, basically wrong. I'll try to dig up a better link. $\endgroup$
    – Glen_b
    Jul 29, 2014 at 23:16
2
$\begingroup$

Use a bootstrap method - resample the points, calculate the difference of medians each time, and look at the distribution to get confidence limits.

$\endgroup$
2
  • $\begingroup$ What if there are very few points? $\endgroup$ Jul 29, 2014 at 20:40
  • $\begingroup$ How many points? The bootstrap will still be a good estimate of the confidence limits - it is based on the actual distribution, rather than making additional assumptions as some statistical tests do. You can always add a small amount of jitter to improve the smoothness of the result. The jitter just needs to be much less than the width of the distribution. For example, if the distribution has a typical width of 1, then when you do the resampling add Gaussian noise of eg $10^{-3}$ to each point. This won't affect your significance test materially. $\endgroup$
    – alpha137
    Jul 30, 2014 at 22:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.