I primarily have a computer science background but now I am trying to teach myself basic stats. I have some data which I think has a Poisson distribution
I have two questions:
Any help would be appreciated. Thanks much
1) What's depicted appears to be (grouped) continuous data drawn as a bar chart.
You can quite safely conclude that it is not a Poisson distribution.
A Poisson random variable takes values 0, 1, 2, ... and has highest peak at 0 only when the mean is less than 1. It's used for count data; if you drew similar chart of of Poisson data, it could look like the plots below:
$\hspace{1.5cm}$
The first is a Poisson that shows similar skewness to yours. You can see its mean is quite small (around 0.6).
The second is a Poisson that has mean similar (at a very rough guess) to yours. As you see, it looks pretty symmetric.
You can have the skewness or the large mean, but not both at the same time.
2) (i) You cannot make discrete data normal --
With the grouped data, using any monotonic-increasing transformation, you'll move all values in a group to the same place, so the lowest group will still have the highest peak - see the plot below. In the first plot, we move the positions of the x-values to closely match a normal cdf:
In the second plot, we see the probability function after the transform. We can't really achieve anything like normality because it's both discrete and skew; the big jump of the first group will remain a big jump, no matter whether you push it left or right.
(ii) Continuous skewed data might be transformed to look reasonably normal. If you have raw (ungrouped) values and they're not heavily discrete, you can possibly do something, but even then often when people seek to transform their data it's either unnecessary or their underlying problem can be solved a different (generally better) way. Sometimes transformation is a good choice, but it's usually done for not-very-good reasons.
So ... why do you want to transform it?
Posting more fun information for posterity.
There is an older post that discusses a similar problem regarding the use of count data as an independent variable for logistic regressions.
Here it is:
Does using count data as independent variable violate any of GLM assumptions?
As Glen mentioned if you are simply trying to predict a dichotomous outcome it is possible that you may be able to use the untransformed count data as a direct component of your logistic regression model. However, a note of caution: When an independent variable (IV) is both poisson distributed AND ranges over many orders of magnitude using the raw values may result in highly influential points, which in turn can bias your model. If this is the case it may be useful to perform a transformation to your IV's to obtain a more robust model.
Transformations such as the square root, or log can augment the relation between the IV and the odds ratio. For example, if changes in X by three entire orders of magnitude (away from the median X value) corresponded with a mere 0.1 change in the probability of Y occuring (away from 0.5), then it's pretty safe to assume that any model discrepancies will lead to significant bias due to the extreme leverage from outlier X values.
To further illustrate, imagine we wanted to use the Scoville rating of various chili peppers ( domain[X] = {0, 3.2 million} ) to predict the probability that a person classifies the pepper as "uncomfortably spicy" ( range[Y] = {1 = yes, 0 = no}) after eating a pepper of corresponding rating X.
https://en.wikipedia.org/wiki/Scoville_scale
If you look at the chart of scoville ratings you can see that a log transform of the raw Scoville ratings would give you a closer approximation to the subjective (1-10) ratings of each chili.
So in this case, if we wanted make a more robust model that captures the true relation between raw Scoville ratings and subjective heat rating, we could perform a logarithmic transformation on X values. By doing this we reduce the impact of the excessively large X domain, by effectively "shrinking" the distance between values that differ by orders of magnitude, and consequently reducing the weight any X outliers (e.g. those capsaicin intolerant and/or crazy spice fiends!!!) have on our predictions.
Hope this adds some fun context!
