2
$\begingroup$

Here it is suggested to use the Hodges-Lehmann estimator. However, I have just read (source) that this estimator is not reliable for skewed distributions. This is the case for the data I have - I know what the population looks like and it is skewed to the right. Aside asymmetric distributions there are several papers criticising the Hodges-Lehman estimator - unsurprisingly because statistical methods have developped since the 1960ties when it was developped.

The other solution is z divided by the root of N (r=Z/SQRT(N)) as described here and here. But what are the assumptions of this formula? random sampling? equal variances? etc?

In this great article, the authors suggest the probability of superiority. "Probability of Superiority is the probability that a randomly sampled score from one population is larger than a randomly sampled score from a second population." So here comes my issue: My data isn't randomly collected and the individual scores in my data set are not independent. My interpretation is that I cannot use this effect size calculation and there is no way round it except collecting new data which isn't feasible. Correct?

What solutions do you reccommend for rank based data?

Up-date: I retrieve historical tweets for category A and B. As a tweet contains more than one word, single words in each tweet depend on each other and are not statistically independent. The words in the tweets are replaced with a score from a database. Accordingly, the scores also depend on each other. The sample sizes are also unequal. The differences between the two categories is normally distributed.

Improve this question
$\endgroup$
8
  • $\begingroup$ "unsurprisingly because statistical methods have developped since the 1960" ... by that criterion, the mean and the median should be even more criticized! At the very least they're hundreds of years old. Can you offer a quote for exactly what criticism was given? In what specific sense is the Hodges-Lehmann estimator unreliable? $\endgroup$
    – Glen_b
    Commented Sep 12, 2016 at 5:42
  • $\begingroup$ Can you describe the way in which you obtained the samples and the manner of the dependence? Your question is really very vague about just the kind of things that might allow people to do something more than shrug. $\endgroup$
    – Glen_b
    Commented Sep 12, 2016 at 5:45
  • $\begingroup$ @Glen_b I have edited my question with the data I am using. Please let me know if you need more information. Re Hodges-Lehmann estimator: the biggest problem is an asymmetric distribution. Since I have such a distribution I didn't read the other 4+ papers talking about other issues but just wanted to flag it for others. $\endgroup$
    – Simone
    Commented Sep 13, 2016 at 12:52
  • $\begingroup$ You already said there was a problem with an asymmetric distribution but what problem is it exactly? What are they saying is wrong, precisely?? $\endgroup$
    – Glen_b
    Commented Sep 13, 2016 at 17:24
  • $\begingroup$ Glen_b my problem is that the estimator is unsuitable for asymmetric distributions. So I cannot use it. I am trying to find a solution to the problem at hand: what effect size can I use for my data? and save the theoretical discussion about the pros and cons of the Hodges-Lehmann estimator for later. Let me know if you want to have the corresponding pages of the book where Wilcox explains why above estimator shouldn't be used with asymmetrically distributed data including the references to the other papers discussing the issue in-depth. $\endgroup$
    – Simone
    Commented Sep 18, 2016 at 9:41

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.