2
$\begingroup$

I'm currently trying to use the rank product statistic on very large datasets but I'm finding that a lot of the entries in my data end up receiving the same rank. To break these ties, I'm randomly assigning the entries the ranks they would have received if they were ranked ordinally. Spearman's rank correlation method takes the average of the ranks, but it also incorporates a correction factor for these cases. Rank product does not seem to have a correction factor of any sort that I can find.

I also can't find any literature or discussions on what the limit is on tied ranks in your data when using rank product. I'm concerned that having significant amounts of tied ranks in my data is causing bias in my results, but I don't know how to go about fixing this issue.

So my question is, how do I handle tied ranks in my data when using rank product? and how many tied ranks are too many?

Any help would be appreciated, thank you!

NOTE: For reference, my datasets contain ~10000 entries and it is not uncommon to see 40 or more entries that would end up being ranked identically.

$\endgroup$
3
$\begingroup$

There is no real limit to the number of ties, with the only complication being that one cannot calculate exact $p$-values. But the way to handle ties is not to randomly assign ranks but rather to use the midrank (average of all the ranks that breaking the ties would create). The midrank is explicitly used in formulas such as Wilcoxon and Spearman, and no correction is required.

$\endgroup$
1
$\begingroup$

More information needed. Here is a guess. Because only 40 or so entries occur out of 10000 or so, I am going to guess that you are using real numbers or a digital approximation of reals. One way around this is to increase the precision. For example, suppose you have $123.456$ as a duplicate product. This clearly has roundoff error, so randomize it to be $123.456\;\pm \;0.0005 \text{ rand}[0,1]$ to as many decimal places as desired whenever $123.456$ occurs.

Now if that is not the problem, say exactly what the problem is.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.