Poisson distribution transformation

Question

I'm quite new to biostatistics so I apologize if my question is too dumb. I'm studying data transformation in biostatistics to fit my data to the normal distribution. I started with the Poisson distribution (which is quite common in the biostatistics: daily admissions, prevalence of rare disease etc) It is recommended to use the square root to fit data to normal distribution. I used stata and this free dataset ( https://www.kaggle.com/datasets/martj42/international-football-results-from-1872-to-2017?resource=download ) with the results of a huge amount of football matches.

I have created a new variable for this dataset, made by the whole amount of goals scored by both teams in each match. You will find that as the independent variable distributed as following:

We can see that the distribution quite approximate the Poisson's one, as confirmed by the values of mean and std deviation.

Then, I've created a new variable with the square root of this variable and the distribution is the following (blue line is how the normal distrib with the same mean and std deviation looks like):

As you can see It's quite far from a normal distribution of my data, as proven by normality tests, but also easily visible from the q-q plot:

So, my question is, why sqrt didn't work? What can I do to transform my dataset to fit the normal distribution?

Answer 1

You did not tell us why you want this, if for regression modeling, it might be a better idea to look into Poisson regression, search this site. But, is your data Poisson distributed? Let us have a look

fitdistrplus::descdist(total, discrete=TRUE, boot=50)  

And the points are clearly away from the line indication Poisson. Probably the reason is that it is overdispersed relative to Poisson:

mean(total)
[1] 2.936
> var(total) 
[1] 4.3665

An often used alternative is the negative binomial, but that doesn't look much better in the plot above ...

The square root transformation stabilizes the variance, but is n ot necessarily best for getting an approximate normal distribution:

fitdistrplus::descdist(sqrt(total), boot=200, obs.pch=8)  

Interestingly, the plot shows almost no variation in the bootstrap distribution of kurtosis/skewness, maybe close to normal?

A transformation good for obtaining approximate symmetry is $\cdot^{2/3}$, let us try:

fitdistrplus::descdist( (total)^(2/3), boot=200, obs.pch=8)   

Both this transformations give higher kurtosis than the normal, basically, the long tail in this data is to heavy to get a very good transformations to normality.

Answer 2

By applying square-root transformation, we try to make distribution normal-like. Poisson's distribution is right-skewed, and square-root transformation compresses higher values so that lower values become more spread out. Resultantly, when we apply square-root transform to Poisson distribution, it becomes a normal-like distribution.

But, the point to note down is that the newly created normal-like distribution should not be compared with normal distribution with mean & varaince of previous Poisson's distribution. Instead, find new mean & variance after applying transformation and then plot the corresponding normal distribution for comparing how far the results are from normal distribution.

Additionally, If you are still unsatisfied with the results of square-root transformation, you can try $2\sqrt{{X+}\frac{3}{8}}$ (Anscombe transformation) or $\sqrt{X}+\sqrt{X+1}$ (Freeman-Tukey Transformation).