I need to do a regression on a variable Y that is skewed right (non normal) and heteroskedastic and therefore violates two assumptions of the normal linear model. The data is non negative (some 0 values) and discrete. enter image description here

As you can see, it is a count variable but it has a very wide range (it counts minutes in increments of ten), which is why I initially excluded a Poisson regression as I thought that given the high counts it would approx a normal distribution (even dividing by ten and the maximum value is 175).

Using a Box-Cox transformation, with a lambda = 0.5, it improves a lot: enter image description here

Now, I have 2 doubts:

  1. I had to translate the data to use the boxcox function in R because of the 0 values, so I don't know if, when I apply the linear model to the square root of Y, I should use the translated or original data set (it gives me a difference in significance for some regressors).
  2. Even using a square root instead of the regular box cox transformation, the interpretation is not as straightforward as I would like (in terms of the explanation of the phenomenon, not just how I would interpret the parameters).

So I was thinking of possibly using a GLM with a Gamma distribution instead of Poisson, but I am not sure if it makes sense with the data I have (discrete and with some 0 values). I am not super familiar with the model so I am unsure, but it is appealing to me as it might have a more useful interpretaion b/c of the multiplicative structure?

Considering all of this, which model is better suited? Should I stick with box-cox or use Poisson or Gamma?

I should note that in the original linear model I had a couple of interactions, so I am not sure it would work with the glms.

Thank you, sorry if my summary was too confused, I have not applied a lot of statistical models to real world data :)

  • 1
    $\begingroup$ As a heads up, we don’t care about the pooled distribution of $Y$. We care about the error term. When we make an assumption about normality in a linear regression, we make it about the error term. Also, that assumption is important for inference but not so much for prediction. $\endgroup$ – Dave Jun 21 at 17:43
  • 1
    $\begingroup$ "Minutes in increments of ten" is not a count variable: it is a discretized duration. Therefore some of your thoughts ought to concern what conditional distribution the actual durations might have (and that would be a continuous distribution). $\endgroup$ – whuber Jun 21 at 17:44
  • $\begingroup$ @Dave Thank you! You are right, I wasn't clear in the question, but i did do a linear model at first and then I tested the assumptions and both homoskedasticity and normality were problems (and i went to box cox from there) $\endgroup$ – Jodie Jun 21 at 17:55
  • $\begingroup$ @whuber thank you! I will consider it. Does that mean that something like a poisson would be incorrect? $\endgroup$ – Jodie Jun 21 at 17:57
  • 1
    $\begingroup$ Goodness of fit tests like Shapiro-Wilk are almost always unhelpful. What is your goal in doing this regression? It may be that the normality assumption doesn’t apply! $\endgroup$ – Dave Jun 21 at 18:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.