Timeline for The variance of the weighted median and optimal weights
Current License: CC BY-SA 4.0
14 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| May 9, 2022 at 12:55 | comment | added | whuber♦ | @Matt The OP only states the medians are asymptotically Normally distributed, not that the underlying distributions are Normal. What remains to be seen--and is generally false--is that a weighted average (or, indeed, any average) of medians is a reasonable estimate of the overall median. | |
| May 9, 2022 at 1:33 | comment | added | user225256 | @whuber, that comment was about summarizing via a weighted average of sample medians, which I was indeed assuming approximately normal, following the OP. | |
| May 8, 2022 at 12:49 | comment | added | whuber♦ | For Normal variables that's the case (because all those quantities in your ratios depend only on the scale parameter), but I have understood this thread to be about random variables generally (at least continuous ones). Indeed, I still don't see where you have stipulated your post applies only to Normal variables. | |
| May 8, 2022 at 1:34 | comment | added | user225256 | @whuber, I claim that if $X,X’$ are normal variables, and $M,M’$ are the medians of $n$ samples of $X,X’$ respectively, then $Var(M)/Var(M’)$, $Var(X)/Var(X’)$, $IQR(M)^2/IQR(M’)^2$ and $IQR(X)^2/IQR(X’)^2$ are all equal. Do you agree, and do you see that as justifying the statement quoted in the above comment? | |
| May 7, 2022 at 21:45 | comment | added | whuber♦ | Re "The variances will be proportional to the squares of the interquartile ranges:" Not so. The variances will also be inversely proportional to the squared densities at the medians. | |
| May 5, 2022 at 17:45 | comment | added | Wolfgang Brehm | Let us continue this discussion in chat. | |
| May 5, 2022 at 17:39 | comment | added | user225256 | I think this warrants a new question: “Suppose distributions $X$ and $Y$ have the same median, and $m$ and $n$ are fixed sample sizes. What weights $v$ and $w$ should be chosen to minimize the variance of the weighted median of $m$ samples of $X$, all weighted by $v$, and $n$ samples of $Y$, all weighted by $w$?” You might also specify whether to assume that $X$ and $Y$ have unknown distributions, unknown symmetric distributions, distributions known up to a translation, or normal distributions with 0, 1, or 2 known parameters. | |
| May 5, 2022 at 17:30 | comment | added | user225256 | It wasn’t clear that the number of samples was fixed. | |
| May 5, 2022 at 17:25 | comment | added | Wolfgang Brehm | Putting all the weight on the sample with the highest density at the median is the same as taking this sample exclusively. So take for example two uniform distributions, one from -1 to 1 and one from -2 to 2. Let's say we have one sample from the first and as many as we like from the second one. Taking the first sample exclusively we have a variance of 1/3 . Even disregarding the best sample and computing the median of just $11$ samples of the other distribution will lead to a variance of about $0.31$, which is lower than 1/3 . Proper weighting gives us $0.22$ . | |
| May 5, 2022 at 16:57 | comment | added | user225256 | I’ve added a comment on the optimal weighted median, which should put the entire weight on one of the samples. | |
| May 5, 2022 at 16:55 | history | edited | user225256 | CC BY-SA 4.0 |
added case of mixture rather than average
|
| May 5, 2022 at 16:20 | comment | added | Wolfgang Brehm | Also, the optimal weights for the weighted median, sadly, are not the same as for the weighted mean, this is easy to show. Tell me your favorite programming language and I'll write you a small numerical demonstration. | |
| May 5, 2022 at 16:18 | comment | added | Wolfgang Brehm | Thank you for the answer, but I'm not looking for the (weighted) average of medians, I'm just looking for the weighted median, no averaging involved. That is for example, take the set of samples with associated weights: {(1,3), (2,4), (3,5), (4,6)} . The weighted median is 3 because $| (5+6) - (3+4+5) | = 1$ is the most equal partitioning according to the given weights you are going to find. | |
| May 5, 2022 at 14:44 | history | answered | user225256 | CC BY-SA 4.0 |