1
$\begingroup$

I want to be able to rank and get percentiles for groups of scores from a range of organisations. These percentiles will be used to judge other organisations.

The organisation's scores are an average of a number of scores within that organisation. The number of scores within an organisation varies from 1 to several hundred.

I want percentiles for many groups of organisations, some of these groups have hundreds of organisations and some have a few as a dozen.

There are clearly some outliers that would skew the percentiles. I'm currently removing them by only looking at those values within 3 weighted standard deviations from the weighted mean (weighted by the number of scores within the organisations, as this reflects the confidence of the organisation's scores). But for large groups this means excluding many.

Is there a better way?

Is it even valid to remove any when calculating percentiles?

$\endgroup$
2
$\begingroup$

The procedure you are using will distort quantiles and result in an estimator that does not estimate something we can understand. Two purposes of using quantiles in the first place are interpretability and robustness. The 0.99 quantile is not affected by "outliers" if there are fewer than a fraction of 0.01 of them but even if 0.02 are strange the 0.99 quantile may still be estimating something that is quite relevant. Note that any approach to removing "outliers" is arbitrary.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.