I have an idea to use ECDF conversion of data to their uniform distribution of equal sample sizes. So, for say 1000 pieces of data, each value should more or less correlate with a .1% representative value, if a value is repeated, that repeated value is translated into another correlated 1/1000%.
So if the number 2 appears 2 times out of 1000 entries.
The overall % of that value is .2%
However, whatever value that was before it, say 1 , that appeared once would be .1%.
so 1 - Maps to .1% and 2 - Maps to .3% (i.e. 2 counts of 2 out of 1000 = .2%, add prior values for cumulative distribution function. aka .1% and .2% = .3%)
I also have a formula that flags for skewed distributions by testing the ECDF converted mean of a distribution for a max error of .275 from .5 mean. If it is, we do a frequency check on the dataset for the median value, and if the frequency is >50% of the values, then we treat the distribution differently. We don't want the 0% to be valued at a high value.
Instead we remove all 0's from the list, convert to ECDF rating. Then re add back 0's. That way we normalize the tail end of the skew.
So would this work to say compare varying sets of data to each other on an additive scale by normalizing them in this method?