1
$\begingroup$

I have 3 variables with the following distributions:

What is the most appropriate transformation to make them as normally distributed as possible?

This data is absolute zero inflated.

$\endgroup$
  • 2
    $\begingroup$ Could you maybe explain a little, why you want these variables to be approximately normally distributed? Secondly: It's difficult to give an answer by just looking at the histograms. But if there are no negative values, you could try Box-Cox-transformations. $\endgroup$ – COOLSerdash Jun 17 '15 at 19:13
  • 2
    $\begingroup$ Assuming that your values are all positive, then logarithms will make the distributions more nearly normal. But as the variables may be percents, then logit transformations may be more appropriate, but only if no value is 100. These distributions look so skewed that no simple, worthwhile transformation could work as well as you wish. Those strong spikes are likely to be flattened slightly but my guess is that some skewness will remain. Conversely, what reason do you have for thinking that normality as such is really important? $\endgroup$ – Nick Cox Jun 17 '15 at 19:16
  • 1
    $\begingroup$ Not the question, but looking ahead to any paper, dissertation or thesis, your program has a quite extraordinary idea about appropriate number of decimal places. Please infer a friendly grin. $\endgroup$ – Nick Cox Jun 17 '15 at 19:23
  • 1
    $\begingroup$ As you said that you want to use these variables in regression models: There are no assumptions about normality in the original variables whatsoever. There is, however, an assumption about the distribution of the residuals (i.e. normality) but even that assumption is not very important. $\endgroup$ – COOLSerdash Jun 17 '15 at 19:26
  • 3
    $\begingroup$ With zeros present, I personally prefer cube roots to $\log(\text{value} + \text{fudge})$. Either will segregate your zeros as a spike. You may prefer to keep data as is. $\endgroup$ – Nick Cox Jun 17 '15 at 23:25
4
$\begingroup$

As you have described your data (i.e. a large number of exactly 0 data points), there is no reasonable transformation to make your data appear normal. By reasonable transformation, I making these two assumptions:

1.) The transformation is deterministic (i.e. if you can't "add noise" to the transformation)

2.) The transformation is monotonic. There are times when non-monotonic transformations, such as taking the absolute value, are "reasonable". But in your case, there is no reason to think that a non-monotonic transformation might be a good idea.

In your first plot, it appears that almost half the values (maybe more?) are equal to 0. Thus, any monotonic, non-deterministic transformation will transform these 0's into either the min or max value. If about half of your (transformed) data points are either the min or the max in a relatively large sample, there is no way your distribution is anything close to normal (excluding degenerate normals, i.e with $\sigma = 0$).

That being said, I'm not sure what your goal is with these datasets, but it seems foolish to attempt to "remove" the zero-inflated aspect from your data, rather than to include this aspect in your analysis. Zero-inflated models are built for exactly this purpose.

$\endgroup$
  • $\begingroup$ I want to run OLS and GWR and include these as explanatory variables $\endgroup$ – I Heart Beats Jun 17 '15 at 20:41
  • 2
    $\begingroup$ Then there's really no problem: non-normal predictors for OLS or GWR is of no concern. $\endgroup$ – Cliff AB Jun 17 '15 at 21:30
  • $\begingroup$ any chance you know of a publication or other documentation, that confirms lack of normality for explanatory variables is acceptable for OLS and GWR? $\endgroup$ – I Heart Beats Jun 18 '15 at 12:53
  • $\begingroup$ Any textbook that describes linear regression or GWR? When they state the model, you'll note that they that the error terms are normally distributed. There are never assumptions about $X$, other than $X^tX$ is full rank. $\endgroup$ – Cliff AB Jun 18 '15 at 14:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.