I am performing an ANCOVA. My dependent variable is the number of days a student is absent from school. As one can imagine, this variable is not normal, so I performed a log10 transformation. When I run the analysis with the transformed variable, I find no significant relationship with my variable of interest. However, when I run it with the original dependent variable the relationship is significant. Also, the homogeneity assumption is violated with the original dependent variable, but not with the transformed variable. Given the violations, I was under the impression that statistical power should be reduced. Why is it that I only find significant results when these assumptions are violated? Can someone help explain this to me? I should also mention that my sample sizes are quite unequal and in this case the smaller cell has a smaller variance on the dependent variable.
-
1$\begingroup$ Looks like you got two answers that hit pretty much all the same points posted within a few seconds of each other. How's that for independent confirmation? $\endgroup$– Glen_bCommented Feb 12, 2013 at 22:35
2 Answers
First, without seeing the data, it is difficult, if not impossible, to tell exactly why you get significance with the model that violates the assumptions but not with the one that doesn't. However, one possible (likely) reason is that you had some students with a large number of absences that also had an unusual value on the independent variable (whatever it is - you could tell us). This would then be an influential point and cause a strange relationship.
Second, it is not the dependent variable that has to be normal, it is the residuals from the model. Have you tested those?
Third, days absent is a count variable. Therefore, unless the mean number of days absent is fairly high, you should not be doing ANCOVA at all, you should be doing some form of count regression. Possibly Poisson but more likely negative binomial regression. If this is over a relatively short time period, then you might even need a zero-inflated model, but it's hard to tell without the data.
Fourth, how did you take the log of the number of days absent? Surely some students had no days absent? Log(0) is undefined.
The dependent variable is not required to have an (unconditional) normal distribution, even if you're using it for inference.
Its conditional distribution should be close to normal unless the sample size is fairly large, or the inference will be somewhat affected.
Note that the linearity assumption and the homoscedasticity assumption are both affected by your transformation - you can't just transform variables without consideration for your other assumptions. Even the additivity of the error term is affected.
Perhaps worst of all -- taking logs of counts will lose all the cases where the count is zero. In some cases this can totally screw up your analysis. Indeed, that loss of points could even be one explanation (of a bunch of possible ones) for the significant/non-significant issue.
You should probably be using a model appropriate for count data, such as a GLM with a binomial response (assuming constant probability of absence within each case), or a Poisson (probably what I'd try first), or quasi-Poisson or even negative binomial.
If you must use linear regression, and see equality of variance as most important, approximate variance-stabilizing transformations include the Anscombe and Freeman-Tukey. These used to be used more often before GLM software became widespread. These all improve the normal approximation quite well as long as the counts tend not to be low.
However, the assumption of linearity is to my mind generally the more central assumption; you must consider whether any transformation is making that worse rather than better.