How to analyse a right skewed dataset

Question

I am trying to analyse several heavily right skewed datasets in order to determine their mode.

The story behind the sets is that they are calculations of performance data from marketing posts. The datasets are the calculations and I am trying to use to determine a baseline for scoring future posts. My theory is that the centre of the mode (after binning the data and creating a histogram) would be the target baseline and new posts will be scored against this baseline to determine a score from 0-100. The reason for the heavily right skewed datasets are largely due to badly performing posts. The tail of these sets can be huge. I am aware of major outliers and have ways to manage that.

The reality is though... I am not a mathematician and this is essentially a side project I am doing my best to see it through. My theory above is as close as I have got to a working model for the scoring system but I'm open to suggestion on better/more accurate statistical frameworks for doing this... I would say though that any suggestions should be explained as if I'm 5.

Thank you in advance for your help and understanding.

Answer 1

The approach I would take is to score new performance data by computing what its percentile would be on your existing data. You can either do this directly or by modeling your data with a probability distribution.

Or, well, because a percentile tells us what percent of our previous data is less than a particular datapoint, we probably want to look at 100-the percentile instead, that way 100 is as good as possible instead of as bad as possible.

I'm going to demonstrate this procedure using python. I'm going to be using these libraries:

import numpy as np
import scipy.stats as stats

The first thing I'm going to do is create some example data (you can skip this step, since you already have some):

# Create some fake data (you already have some so can skip this :)).
old_data = stats.gamma.rvs(5, scale=22, size=1000)
new_data = np.array([50, 80, 100, 200])

Here's the first approach, of manually computing percentiles. We just calculate what percent of the existing performances we exceed:

# Approach 1: Empirical Percentiles.
for i in range(len(new_data)):
    print(100-np.mean(new_data[i] > old_data) * 100)

The result of this is

91.5
68.1
49.4
5.5

If you fancy it, you can use a parametric distribution instead. I'm trying the Gamma distribution out:

# Approach 2: Model-based percentiles.
# Step 1: Find a Gamma distribution that fits your data 
alpha, _, beta = stats.gamma.fit(data, floc = 0.)

# Step 2: Use that distribution's CDF to get percentiles.
scores = 100-100*stats.gamma.cdf(new_data, a = alpha, scale=beta)
print(scores)

The result of this is:

[92.38913684 70.38297238 52.46867026  4.86655548]

It would have to be a rather precocious 5 year old who understands this, so feel free to fire off questions.